Skip to content

Public security notice: This documentation is intentionally redacted. Sensitive server paths, private keys, secret tokens, and origin network details are removed.

Token Data Viewer

The Token Data Viewer is a CSV-backed snapshot of market data used as both:

  • a fast baseline for many metrics during assessments, and
  • a signal for when it is worth running a full risk assessment.

Files and locations

The viewer uses the following files under DATA_DIR:

  • token_data_viewer.csv – primary snapshot used by the main script.
  • token_data_viewer_export.csv – alternate / export view for other tools.

The engine only reads these files. The legacy desktop helpers that used to generate them (update_token_data_viewer.py, refresh_token_data_viewer.py, comprehensive_data_fix.py, fix_liquidity_and_apis.py and similar) were removed: they could only run on one machine and nothing imported them.

Loading the snapshot

Early in the main script (scripts/engine/defi_complete_risk_assessment_clean.py), _load_token_viewer_snapshot() reads the CSV:

python def _load_token_viewer_snapshot(): """Load token_data_viewer.csv into chain-aware symbol metrics.""" viewer_path = os.path.join(DATA_DIR, "token_data_viewer.csv") snapshot: Dict[str, Dict[str, Any]] = {} if not os.path.exists(viewer_path): return snapshot try: df = pd.read_csv(viewer_path) for row_pair in df.iterrows(): row = row_pair[1] sym = str(row.get("Symbol", "")).upper() ... entry = { "market_cap": _num(row.get("Market Cap", 0)), "volume_24h": _num(row.get("Volume 24h", 0)), "liquidity": _num(row.get("Liquidity", 0)), "price": _num(row.get("Price", 0)), "holders": _num(row.get("Holders", 0)), "source": "token_data_viewer" } snapshot[sym] = entry if chain: snapshot[f"{chain}:{sym}"] = entry

The helper _num cleans string / numeric fields, coercing invalid or NaN values to 0.0, ensuring downstream code does not crash on malformed input.

Using the snapshot during scoring

Throughout the engine, the viewer snapshot is treated as:

  • an additional non-estimated seed when cache/fallback data is absent, and
  • a last-resort liquidity / market data source when live APIs are temporarily unavailable.

For example, when computing liquidity and market-structure metrics, if live sources and cache are missing, the engine may fall back to viewer-derived liquidity values while clearly marking them as such in the provenance data.

Because entries are keyed by both SYMBOL and chain:SYMBOL, the snapshot can distinguish between the same ticker across different chains (e.g. USDC on Ethereum vs Polygon).

Updating the viewer

No generator for these files ships with the suite any more. When token_data_viewer.csv is absent the snapshot is simply empty, and the engine relies on live APIs and its caches, which is the normal production path. A file placed in DATA_DIR by hand is still read, with the columns shown above.

Operational best practices

  • Treat sparse viewer data as a signal
    If many rows show 0 or N/A, consider delaying large runs or restricting them to well-covered assets.

  • Use viewer data as a baseline, not as truth
    Viewer values are convenient and fast, but where possible they should be superseded by live API calls and caches at assessment time.

By using the Token Data Viewer as a first-stage market snapshot and quality gate, the engine avoids wasting rate limits on illiquid or inactive tokens and provides more stable, timely risk reports.