搜索“RAG”

Authors Shaun Sutner Shaun Sutner, a journalist with more than 25 years of daily newspaper experience and 12 years at Informa TechTarget as an editor and writer, directs news coverage for AI Business. He was previously a senior news and features writer covering health IT and HR software at TechTarget and a senior news director overseeing coverage of AI, business analytics, data management and government tech regulation. Sutner's newspaper career included investigative reporting and covering the

匹配字段正文

he said. Shaun Sutner Shaun Sutner, a journalist with more than 25 years of daily newspaper experience and 12 years at Informa TechTarget as an editor and writer, directs news coverage for AI Business. He was previously a senior news and features writer covering health IT and HR software at TechTarget and a senior news director overseeing coverage of AI, business analytics, data management and government tech regulation. Sutner's newspaper career included investigative reporting and covering the

匹配字段正文

文 | 穷奇观察 2026 年 10 月 8 日，Anthropic 一天之内推出两项“免费”安排。 第一项叫“关键基础设施防御计划”（Critical Infrastructure Defense Program，CIDP）。前沿 Claude 模型、驻场工程师和威胁研究，交给 11 家创始伙伴：埃森哲、博思艾伦、CrowdStrike、德勤、Dragos、日立、Insane Cyber、Nozomi Networks、Palo Alto Networks、普华永道、罗克韦尔自动化。 这些伙伴服务的对象，是电网、水务、交通网络，以及政府系统背后的运营技术（OT）。 第二项叫 OSS Scanner。免费、自愿接入，用 Anthropic 最强的模型（包括 Claude Mythos）给开源项目做定期安全扫描。 两项归在同一个名字底下：Anthropic Cyber Mission。 把这两件事并排放，有一处别扭：一家卖模型的公司，同时来当开源世界的审计者、关键基础设施的守门人，以及“什么算漏洞”的裁定者。 这篇文章不判它想不想作恶。只判一件事：这几个身份叠在同一家公司身上，结构上哪里

匹配字段正文

to 30,556 after the open. Micron Technology slid 4.8%, Advanced Micro Devices dropped 1.4%, while Nvidia lost 0.8% and Broadcom slipped 0.2%. However, the Dow Jones industrial average, which tracks 30 large US companies, rose nearly 0.4% in early trading, while the FTSE 100 index in London hit a three-week high on the back of gains in ‌financial stocks, retreating oil prices and easing bond yields. The UK’s leading share index rose 0.73% to 10,762. Brent crude, the global oil benchmark, dropped

匹配字段正文

recomputation; Satellites pull core weights through Mooncake’s Transfer Engine and join an ongoing batch; Streaming training lets cells claim micro-steps as data becomes ready; Average role switches took 8.52 s (rollout to training) and 3.46 s (training to rollout) How does the Trajectory Processor build training samples? A proxy records exact token IDs and behavior log-probabilities using token-in, token-out (TITO). Calls are stored as prefix-shared trajectory trees. Partial scoring: Timed-out

匹配字段正文

he core Garçon interface One typically uses Garçon by launching a new server. Launching a server requires only a name for the server, and a path to the model in our distributed storage; all other details such as model configuration and the number of GPUs are autodetected: Once a server is launched, you can connect to it, either from an interactive notebook, or from scripts for automated analyses: (We use the “r” prefix by convention to denote a “remote” model running via garçon). Servers can als

匹配字段正文

kenizers typically merge spaces onto the start of words. But occasionally a word will appear in a context where there isn't a space in front of it, such as at the start of a new paragraph or after a dialogue open quote. These cases are rare, and as such, the tokenization isn't optimized for them. So for less common words, it's quite common for them to map to a single token when a space is in front of them (" Ralph" → [" Ralph"]) but split when there isn't a space ("Ralph" → ["R", "alph"]). It's

匹配字段正文

nsformers can be adapted to do many useful things without re-training, such as translation, question-answering, arithmetic, and many other tasks. Using “prompt engineering” to leverage in-context learning became a popular topic of study and discussion. At least two importantly different ways of conceptualizing and measuring in-context learning exist in the literature. The first conception, represented in Brown et al., focuses on few-shot learning of specific tasks. The model is prompted with sev

匹配字段正文

ith their basis dimensions. This is because the activation functions in some sense make these directions natural and useful. Just as a CPU having operations that act on bytes encourages information to be grouped in bytes rather than randomly scattered over memory, activation functions often encourage features to be aligned with a neuron, rather than correspond to a random linear combination of neurons. We call this a privileged basis. Having features align with neurons would make neural networks

匹配字段正文

tures to become interpretable, but it may also make it even harder to interpret some already non-interpretable features. On balance, however, it still seems like a win in that it pragmatically increases our understanding. Architecture affects polysemanticity and MLP interpretability. Although it isn't a perfect solution, SoLU is a proof of concept that architectural decisions can dramatically affect polysemanticity, making it more tractable to understand transformer MLP layers. This suggests tha

匹配字段正文

t in other cases don't? We hypothesize that there are really two countervailing forces driving this: Privileged Basis: Only some representations have a privileged basis which encourages features to align with basis directions (i.e. to correspond to neurons); Superposition: Linear representations can represent more features than dimensions, using a strategy we call superposition. This can be seen as neural networks simulating larger networks. This pushes features away from corresponding to neuron

匹配字段正文

he model by either {R} (if the matrix is reading from the residual stream) or R^\intercal (if the matrix is writing to the residual stream). Random rotations do tend to hurt on average, but the numbers are absolutely tiny (for scale, the baseline model has a test loss of about 3.02 nats) and we see much larger noise from e.g. different random seeds at model initialization. From this, we conclude that this model is genuinely rotation-invariant, and that this fact holds even when accounting for fl

匹配字段正文

takes more than one layer to retrieve a feature from superposition, this seems pretty unfavorable since it both wastes compute and (by storing an intermediate result) the very "storage space" that superposition was trying to save. As a result, typical dictionary learning algorithms may be searching over too broad a space of possibilities. One special concern is that they might hallucinate "extra features", which are pairs of features and allow for sparser representations, but don't make sense fr

匹配字段正文

ribed three strategies to finding a sparse and interpretable set of features if they are indeed hidden by superposition: (1) creating models without superposition, perhaps by encouraging activation sparsity; (2) using dictionary learning to find an overcomplete feature basis in a model exhibiting superposition; and (3) hybrid approaches relying on a combination of the two. Since the publication of that work, we've explored all three approaches. We eventually developed counterexamples which persu

匹配字段正文

our findings about feature splitting. But an important open question remains whether we can find evidence for a "strong superposition hypothesis", or whether features are just a pragmatically useful abstraction. Features in an 8-layer Model In Towards Monosemanticity, we highlighted scaling dictionary learning to larger models as a key direction of future work. We have since applied dictionary learning to the MLP activations of an 8-layer transformer, and identified a number of interesting feat

匹配字段正文

etter ways to handle dying neurons. Tanh Penalty in Dictionary Learning One challenge in training sparse autoencoders with an L1 penalty is shrinkage. That is, in addition to encouraging sparsity, an L1 penalty encourages the autoencoder activations to be smaller than they would be otherwise. In our experiments, we came to believe that this was at least partly responsible for our sparse autoencoders recovering a smaller fraction of the MLP loss than we might have hoped. To remedy this, we studie

匹配字段正文

ink, or that hide the core complexity we’re trying to study. So while there are good reasons that mature fields lean heavily on summary statistics, and develop a culture that encourages them, that culture can be misleading in pre-paradigmatic fields, and in interpretability we need to be mindful of this. Rigor and The Signal of Structure How do we know if qualitative results are "real"? What does rigorous qualitative research look like? We don't have tools like "statistical significance" to fall

匹配字段正文

es cancellation between direct and indirect effects. The paper works around this by running one backwards pass per layer, skipping that layer when computing gradients, and then averaging the effects across the passes They validate their improvements by comparing their approximations to nodes' influences with the "ground truth" of activation patching, on Pythia-12B. They also demonstrate that attribution patching is more efficient than some other clever ways of approximating activation patching.

匹配字段正文

r-running tasks. That kind of delegation requires more than increasingly capable models. Related:Google’s workplace agent is out, but questions abound An AI system that drafts a paragraph needs relatively little organizational context. An agent expected to complete work across multiple applications is different. To do that, it may need access to company data, an understanding of business processes, memory of previous work and permission to take actions in enterprise systems. Those requirements h

匹配字段正文

t the company is prioritizing “higher-value unit placements, repricing for higher commodity costs” to offset the lower volume of PC shipments. “We continue to expect memory and storage costs to increase further as a percentage of the bill of materials,” Parkhill said. Enterprises picked up new PCs earlier in anticipation of price increases and are now pulling back, said analyst Jack Gold of J. Gold Associates. That’s partially because PC makers are passing higher memory and component costs on to

匹配字段正文