One way to address the impact of context degradation in generative AI sessions, recently demonstrated by Evans’ Law, is to track how close you are to thresholds. Is this easy to do today? It’s a complicated answer with a relatively simple fix.
- On most public chat interfaces (ChatGPT, Claude, Gemini, Copilot, etc.) – NO
No — you cannot ask the model for your live session token count, because the model itself doesn’t have access to that runtime data.
Here’s why:
- Token counting happens outside the model, in the API or front-end layer (where requests are batched, truncated, or streamed).
- Once the model starts generating, it no longer “sees” how many tokens it has consumed — it’s blind to its own context length.
- For security and performance reasons, that metadata (token totals, entropy, memory load, etc.) is not passed back to the model.
If you ask ChatGPT or Claude, “How many tokens are in this session?”, you’ll get an estimate, not an accurate count — based on text length or heuristics.
2. On developer APIs (OpenAI, Anthropic, Mistral, etc.) – YES
Yes — developers can track it precisely via:
- Token usage fields returned in API responses (e.g., usage.total_tokens, usage.prompt_tokens, usage.completion_tokens).
- Client-side counters using SDK utilities like tiktoken (OpenAI) or anthropic.count_tokens().
That’s how platforms could power a “Coherence Health Meter” or “Context Budget Gauge”: the data exists, but isn’t exposed yet to end users in real time.
3. In enterprise deployments
If your company is running an internal LLM or API gateway (e.g., Azure OpenAI, Bedrock, Vertex AI, or a private endpoint), your admins can enable live telemetry — tracking:
- Average context load per user or per query,
- Response entropy and attention drop-off rates,
- “Coherence health” as a percentile of max safe context.
These setups could easily provide dashboard-style token counters and coherence alerts — which is exactly what Evans’ Law suggests platforms need to implement to protect users from silent degradation.
4. For users right now
If you’re a non-technical user and want to approximate your token use:
- Assume 1 token ≈ 4 characters of English text, or ¾ of a word.
- So a 1,000-word conversation roughly equals 1,300–1,500 tokens.
- If your platform supports 128K context, 70% of that (safe zone) is ~90,000 tokens — or around 60,000–70,000 words total context.
In practice, drift begins well before that depending on how dense your prompts are.
You can use free tools like tiktokenizer.com or OpenAI Tokenizer Playground to estimate token count by pasting your text.
5. What to ask for
Until these coherence tools exist, users should push providers to add:
- “Show token usage” toggles per chat,
- Soft alerts at 70% context,
- Auto-summarize on overflow, and
- Optional “Coherence Health” meters.
It’s a trivial implementation for any provider with API telemetry — it’s just not user-visible yet.
