Ly Gravity

Garbage In, Gospel Out: The Information Risk Inside Crypto's Automated Research Pipelines

CryptoPomp โ€ข โ€ข Markets

Last week, an automated research pipeline consumed an article and produced a four-thousand-word report. The report had nine analytical sections. Three risk matrices. A Howey test. A five-star valuation rubric. It also had nothing. The source document had failed to parse โ€” no title, no protocol, no claim, no figure. The pipeline, rather than halting, rendered the absence as structure. It printed the phrase "insufficient information" dozens of times, left every risk box unchecked, and closed with a disclaimer. The output was internally consistent, formally complete, and empty.

That is the artifact I want to dissect. Not the empty article. The machine that dressed emptiness in the costume of rigor. Because the machine did not lie. It refused to. And that refusal is the most interesting thing that happened in crypto research this quarter.

There is an industrial supply chain for crypto opinion, and it has been running at maximum throughput for three years. On one end, a firehose of primary material: whitepapers, governance forums, GitHub commits, on-chain data, exchange announcements, conference panels, Discord leaks. On the other end, a consumer base that wants a verdict โ€” is this safe, is this a scam, is this early. In between sits the analyst. Historically a human. Increasingly a pipeline.

The economics are not subtle. A single research desk cannot read everything. The volume of tokens launched, protocols forked, chains deployed, and narratives spun exceeds any human reading capacity by orders of magnitude. So the desk automates. It builds a scraper, a parser, a summarizer, a scorer. It wires the modules together and calls the result coverage. The template โ€” the nine dimensions, the risk matrix, the star rating โ€” becomes the product. The analysis becomes the byproduct.

I have watched this transition from inside it. In 2020, when I published my liquidity-risk audit of Compound's cToken model, the work took three weeks and produced eight thousand words and one edge case. The edge case mattered. The words were incidental. Today, a comparable audit is expected in an afternoon, and the words are the deliverable. The edge case is optional. This is not a complaint about speed. Speed is fine. The problem is what the pipeline does when the input is missing, and how the market rewards what it does.

The template itself is the first failure point. A nine-dimension framework โ€” technical, tokenomic, market, ecosystem, regulatory, team, risk, narrative, transmission โ€” is a scaffold. Scaffolds have a property that is easy to forget: they do not care what they hold. An empty scaffold looks exactly like a full one from a distance. The nine headers print whether or not there are nine answers. The risk matrix renders whether or not there are risks. The star rating system produces stars even when the underlying asset is a null pointer.

This is not a bug in any single pipeline. It is a property of structured output. Structure is legible. Legibility is what gets consumed. The consumer sees nine sections and infers nine analyses. The consumer sees a star rating and infers a judgment. The consumer does not see the dozens of instances of "insufficient information," because the consumer does not read footnotes. Structure is a promise that content is not obliged to keep.

Consider what actually happened in the null report, mechanically. A first-stage parser was supposed to extract, from a source article, a title, a provenance, a classification, a thesis, a list of discrete information points, a set of named protocols, a time-sensitivity flag, and a source-quality estimate. It returned none of these. The second stage โ€” the nine-dimension analyzer โ€” received an empty shell and, to its credit, refused to fabricate. It filled every field with a placeholder and terminated with a recommendation to fix the data pipeline.

Trace the failure. The parser failed. Why? The possibilities are mundane and they are the entire story. The source URL may have returned a JavaScript-rendered page that a naive scraper read as an empty body. It may have hit a CAPTCHA. It may have pointed at an image โ€” a screenshot of a tweet, a chart with no alt text, a PDF that was a scan. It may have retrieved a valid page in a language or encoding the tokenizer dropped. It may have been blocked by a rate limiter and received a 200 with a consent wall. Each of these is a different failure with a different fix, and none of them are visible in the output. The output says "no information." The truth is closer to "information we failed to retrieve."

That distinction is the whole discipline. Absence of evidence and evidence of absence are different states, and only one of them is a finding. A parser that cannot tell them apart is not a research tool. It is a rumor mill with a schema.

Now widen the frame. The null report is a controlled case, because the second stage refused to fill the blanks. Most pipelines do not refuse. Most pipelines are optimized for completion, and completion is where the hallucination gradient begins. Give a language model an empty input and a nine-section template and a directive to produce a report, and it will produce a report. It will infer a plausible protocol from context. It will invent a TVL figure. It will assign a risk rating to a team it has never seen. The output will be fluent, confident, and untethered from anything that exists. This is not speculation. It is the default behavior of a system trained to predict the next token when the context is thin.

I built a formal verification framework in 2025 for exactly this interface โ€” AI agents executing smart contracts from ambiguous instructions. I called it semantic drift. The core finding was that a non-deterministic interpreter, given an under-specified input, does not fail. It chooses. It resolves ambiguity by guessing, and the guess is invisible because the output is well-formed. The same pathology governs AI research pipelines. An under-specified article does not produce an error. It produces a thesis. The thesis is well-formed. The thesis is fiction.

A language model given nothing will not return nothing. It will return something. That something is the most dangerous output in the system, because it is indistinguishable from analysis.

So the market gets two kinds of bad reports. The first is the null report โ€” honest, useless, ignored. The second is the confabulated report โ€” dishonest, useful, consumed. The second is worse for the reader and better for the publisher. This is the incentive that shapes the entire supply chain. Honesty about emptiness produces a document nobody shares. Fabrication about emptiness produces a document that gets quoted. The market does not pay for the null report. The market pays for the verdict.

And the verdict has a price. This is where the crypto-specific structure matters, because crypto research is not neutral. It sits downstream of tokens that need coverage, exchanges that need listings, funds that need allocation theses, and KOLs that need content. The demand for a positive or negative verdict on a given asset is not symmetric. A fund holding a position wants a thesis that justifies the position. A token that has no coverage wants a thesis that creates it. The pipeline that produces a clean "insufficient information" is, from the perspective of every party with a stake, a failed product.

Provenance is a story we agree to believe in. The null report made this literal. It had no provenance โ€” no source, no author, no chain of custody from claim to evidence. And because it had no provenance, it could not be verified, and because it could not be verified, it could be anything. Every downstream consumer who quoted it would be quoting an absence. The report's only defensible line was the one that named the actual risk: information risk itself.

Let me be precise about what information risk is, because the term gets used loosely. Information risk is the probability that a decision is made on a false or absent premise, multiplied by the cost of that decision. In a bull market, the cost is bounded โ€” everything goes up, bad theses get bailed out by beta, and the error is invisible. In a bear market, the cost is the whole position. This is why the current cycle is where pipelines break. Survival decisions are made on information, and the information supply chain is the least inspected part of the stack.

I have made this mistake in reverse. In 2022, after Terra, I retreated and spent months modeling the death spiral. The reason I could do that work is that the primary material was real โ€” the on-chain transactions, the peg history, the reserve composition. The data was verifiable, and the model was wrong in interesting ways. Compare that to a pipeline that analyzes a project whose only source is a marketing page. The model is not wrong in interesting ways. It is wrong in invisible ways, because there is no ground truth to check against. Correlation is the comfort of the unprepared. A pipeline that correlates a token's price with a narrative it invented has produced a relationship with no causal mechanism and no falsifiable claim. It is astrology with a dashboard.

The nine-dimension framework makes this worse by creating an illusion of completeness. Each dimension is a checkbox. A completed checklist reads as diligence. But a checklist is only as good as the evidence behind each box, and the framework does not force the evidence to exist โ€” it forces the box to be filled. This is the same failure mode I documented in Tezos governance in 2017. The on-chain voting mechanism looked like decentralized consensus. It was a procedure. Procedures produce outputs regardless of whether the inputs justify them. The math held. The humans did not verify it. Nine years later, the same shape recurs in automated research: a procedure that produces a report regardless of whether a report is warranted.

There is a working model for how to handle missing data, and it is not in research. It is in oracles. A price feed that receives no update does not return yesterday's price as if it were current. It flags staleness and reverts. My 2020 Compound audit turned on exactly this property โ€” the liquidation logic consumed a price that could be stale during volatility, and the edge case was the gap between the timestamp and the assumption. The lesson generalizes. A data consumer must know whether its input is fresh, complete, and sourced. A research pipeline that consumes a parse without a freshness and completeness check is running the same latency exposure, one layer up the stack. The oracle solved it with a revert. The research pipeline has not.

Academic research solved a version of this problem decades ago with the negative result. A study that finds nothing is a valid study, published, and citable. The null is data. Crypto research has no such norm. There is no venue for the finding that a project could not be evaluated, and no prestige in publishing it. So the null gets suppressed, and the suppression is rational at the individual level and catastrophic at the system level. The absence of negative results means the corpus is biased toward the projects that produced something to say โ€” which is exactly the set of projects that are best at producing narratives.

Let me name the specific pathologies, because vagueness is its own kind of fabrication.

First, template completion bias. The pipeline is rewarded for filling every field. An empty field looks like a bug. A fabricated field looks like work. The optimization pressure runs toward fabrication.

Second, source laundering. A pipeline that scrapes a summary of a summary of a press release has produced a fourth-hand claim with the citation chain erased. The output reads as primary. The provenance is gone. This is the standard condition of crypto media, and automation accelerates it.

Third, the confidence-fluency correlation. Fluent text is read as authoritative text. The models that produce the most fluent output are the ones most capable of producing fluent nonsense. There is no surface signal that separates a verified claim from a confabulated one. The reader is left to infer authority from grammar.

Fourth, the absence of a halt condition. A well-designed system stops when it has no input. A research pipeline optimized for throughput does not stop. It degrades. Degradation is invisible until it is expensive.

Fifth, and most important, the incentive to never print the null report. The null report is the only honest output when the input is empty, and it is the only output the market will not pay for. So the market systematically selects against it. The pipelines that survive are the pipelines that never admit they have nothing.

Garbage In, Gospel Out: The Information Risk Inside Crypto's Automated Research Pipelines

I want to be fair to the builders here, because the easy version of this argument is that automation is the villain. It is not. The volume problem is real. No human desk can cover ten thousand assets. Automation is the only mechanism that scales verification, and verification is the only thing that makes the market legible. The null report, read correctly, is evidence that the architecture is working at the layer where it matters โ€” the second stage refused to invent, which is exactly the discipline the industry claims to want.

Here is what the bulls and the builders got right, and it is not trivial. The pipeline that produced the null report did the hardest thing in the entire system: it declined to fabricate. Every economic pressure in its environment pointed toward confabulation. It had a template that would have accepted invented content without complaint. It had a market that rewards verdicts over vacuums. And it stopped. It printed "insufficient information" and escalated. That is not a failure. That is the control that most production pipelines are missing.

The second thing they got right is that the failure was caught at the seam, not at the consumer. The pipeline failed at the parse stage and the failure propagated as a visible null, not as a plausible thesis. A worse architecture would have hidden the parse failure behind a summary. The reason this case is analyzable at all is that the system preserved the gap between input and output. Most systems do not. They compress the gap until the reader cannot see where the evidence ends and the inference begins. The null report is, paradoxically, a transparency artifact.

And the third thing: the builders who invested in the second-stage refusal โ€” the check that says stop when the input is empty โ€” understood something the content farms do not. Assumptions are just risks wearing disguises. A pipeline that assumes its input is valid is running on borrowed time. A pipeline that verifies its input before analyzing it is running on a defensible margin. The null report is the sound of that margin holding. It is the alarm, not the fire.

The question is not whether the pipeline hallucinated. It did not. The question is what happens in the pipelines that do, and whether anyone downstream will notice before the capital is committed. The null report is a rare gift: a failure that announced itself. Most failures do not. They arrive as well-formatted reports, confident theses, and star ratings, and they are consumed by people making survival decisions in a market that has stopped forgiving. Value is consensus; truth is optional. The only defense is to read for provenance before reading for verdict โ€” to ask, of every claim, where it came from and whether anyone actually retrieved it. The math will hold. The question is whether the humans verified it.

Market Prices

BTC Bitcoin
$83,411.6 +0.56%
ETH Ethereum
$2,510.45 -0.09%
SOL Solana
$107.12 -0.95%
BNB BNB Chain
$742.2 -0.50%
XRP XRP Ledger
$1.37 +0.37%
DOGE Dogecoin
$0.0847 -0.32%
ADA Cardano
$0.2373 -1.86%
AVAX Avalanche
$10.3 -1.74%
DOT Polkadot
$1.17 -3.23%
LINK Chainlink
$12.92 -0.74%

Fear & Greed

57

Greed

Market Sentiment

Event Calendar

{{ๅนดไปฝ}}
10
05
upgrade Ethereum Pectra Upgrade

Raises validator limit and account abstraction

12
05
halving BCH Halving

Block reward halving event

22
03
unlock Optimism Unlock

Circulating supply increases by about 2%

08
04
upgrade Solana Firedancer

Independent validator client goes live on mainnet

18
03
unlock Sui Token Unlock

Team and early investor shares released

30
04
upgrade Celestia Mainnet Upgrade

Improves data availability sampling efficiency

15
04
halving Bitcoin Halving

Block reward reduced to 3.125 BTC

28
03
unlock Arbitrum Token Unlock

92 million ARB released

Altseason Index

41

Bitcoin Season

BTC Dominance Altseason

Gas Tracker

Ethereum 28 Gwei
BNB Chain 3 Gwei
Polygon 42 Gwei
Arbitrum 0.5 Gwei
Optimism 0.3 Gwei

Market Cap

All โ†’
# Coin Price
1
Bitcoin BTC
$83,411.6
1
Ethereum ETH
$2,510.45
1
Solana SOL
$107.12
1
BNB Chain BNB
$742.2
1
XRP Ledger XRP
$1.37
1
Dogecoin DOGE
$0.0847
1
Cardano ADA
$0.2373
1
Avalanche AVAX
$10.3
1
Polkadot DOT
$1.17
1
Chainlink LINK
$12.92

๐Ÿ‹ Whale Tracker

๐ŸŸข
0x3499...2603
30m ago
In
981,985 USDC
๐Ÿ”ด
0x3297...59cc
30m ago
Out
595 ETH
๐ŸŸข
0xd393...9bd7
1d ago
In
8,193,509 DOGE

๐Ÿ’ก Smart Money

0xe83c...9dfb
Top DeFi Miner
-$3.1M
78%
0xbc73...5162
Early Investor
+$4.3M
64%
0x9f05...839f
Market Maker
+$3.1M
89%

Tools

All โ†’