NAll news

All news

Sunday2 items

Friday4 items

Thursday2 items

Wednesday1 items

Tuesday2 items

  2. Hacker News TrendingOpinion
    AI score63

    AI Text Watermarking and Quality Loss: SynthID-Text Weakens Response Diversity in Multi-Turn Generation

    Keyvan's blog post argues that the claim in the Anthropic and Google SynthID-Text papers that "watermarking does not affect output quality" is based only on tests of single responses and does not cover multi-turn conversations or multiple-retry scenarios. The article cites the "diversity/detectability trade-off" in the SynthID-Text supplementary materials, pointing out that watermarking makes multiple responses to the same prompt more similar, potentially reducing the overall quality of generate-then-select systems. The article also cites a reproduction experiment by technical consultant Natzir Turrado: the watermarked model described a trip to a seaside town five times, with four openings nearly identical, while the unwatermarked model produced three different openings.

    #English text is being prepared.#Safety / alignment#Anthropic

Monday2 items

Sunday2 items

Saturday1 items

Friday5 items

  2. jietangOpinion
    AI score25

    English translation is being prepared.

    Quoted postLiam Fedus · @LiamFedus

    An excellent history of scaling laws from @jietang. In 2020, we explored the limits of sparsity in Switch Transformers by routing each token to only 1 out of 2048 experts (in retrospect, a bold choice). The model had fewer than 3B activated parameters, but 1.6T total parameters (comparable to today's frontier models). The 1.6T model achieved better C4 perplexities than the T5 models using far less compute, set a new SOTA on TriviaQA, but was dumb as bricks on reasoning tasks like SuperGLUE. The lesson was that the optimal tokens-per-parameter ratio is highly task-dependent. Or as @NShazeer had already intuited: FLOPs were intelligence; parameters were knowledge!

    #English text is being prepared.#Reasoning#English text is being prepared.