• @drspod@lemmy.ml
    link
    fedilink
    542 months ago

    I thought that guardrails were implemented just through the initial prompt that would say something like “You are an AI assistant blah blah don’t say any of these things…” but by the sounds of it, DeepSeek has the guardrails literally trained into the net?

    This must be the result of the reinforcement learning that they do. I haven’t read the paper yet, but I bet this extra reinforcement learning step was initially conceived to add these kind of censorship guardrails rather than making it “more inclined to use chain of thought” which is the way they’ve advertised it (at least in the articles I’ve read).

    • @iii@mander.xyz
      link
      fedilink
      English
      45
      edit-2
      2 months ago

      Most commercial models have that, sadly. At training time they’re presented with both positive and negative responses to prompts.

      If you have access to the trained model weights and biases, it’s possible to undo through a method called abliteration (1)

      The silver lining is that a it makes explicit what different societies want to censor.

      • Snot Flickerman
        link
        fedilink
        English
        5
        edit-2
        2 months ago

        Hi I noticed you added a footnote. Did you know that footnotes are actually able to be used like this?[1]

        Code for it looks like this :able to be used like this?[^1]

        [^1]: Here's my footnote


        1. Here’s my footnote ↩︎

        • Farid
          link
          fedilink
          112 months ago

          Do you mean that the app should render them in a special way? My Voyager isn’t doing anything.

          • Snot Flickerman
            link
            fedilink
            English
            52 months ago

            I actually mostly interact with Lemmy via a web interface on the desktop, so I’m unfamiliar with how much support for the more obscure tagging options there is in each app.

            It’s rendered in a special way on the web, at least.

            • CodexArcanum
              link
              fedilink
              English
              72 months ago

              That’s just markdown syntax I think. Clients vary a lot in which markdown they support though.

              • Snot Flickerman
                link
                fedilink
                English
                52 months ago

                markdown syntax

                yeah I always forget the actual name of it I just memorized some of them early on in using Lemmy.

    • thisisbutaname
      link
      fedilink
      152 months ago

      I saw it can answer if you make it use leetspeak, but I’m not savvy enough to know what that tells about guardtails