GraffereneSample
Measurement report
This is the report a pilot ends with, unedited, from a measure-only run of a 34-request test trace against a vLLM server. The trace was built to repeat, so its percentages describe the trace, not your traffic. In a pilot the same report covers a week of your requests.
Window: 2026-09-25 23:59 to 2026-09-26 23:59 (UTC).
| Count | Share of requests | |
|---|---|---|
| Requests | 34 | |
| Upstream calls | 34 | 100.0% |
| Would have been served: exact / semantic | 8 / 5 | 38.2% |
| Semantic would-be hits checked by a verifier | 0 |
| Upstream tokens | Spent | Would have been saved |
|---|---|---|
| Prompt | 36714 | 14458 (39.4%) |
| Completion | 980 | 341 (34.8%) |
Similarity of semantic hits
| Cosine | Count |
|---|---|
| 0.00–0.90 | 0 |
| 0.90–0.92 | 0 |
| 0.92–0.94 | 1 |
| 0.94–0.96 | 3 |
| 0.96–0.98 | 1 |
| 0.98–1.00 | 0 |
By day
| Date | Requests | Upstream calls | Served | Would serve |
|---|---|---|---|---|
| 2026-09-26 | 34 | 34 | 0 | 13 |
What these numbers do not show
- 5 semantic would-be hits were not checked by a verifier. A would-be hit is a question the cache would have answered with a stored answer to a similar question, and similar is not the same: on labeled lookalike questions (QQP) about one cosine-only semantic hit in five at 0.92 answered a different question. Review the sample before counting semantic savings.
- Measure-only mode counts concurrent identical requests as misses (serving mode would coalesce them into one upstream call), so bursty traffic is undercounted.
Review sample
Semantic would-be hits, least similar first. Would the stored answer to the right-hand question have been a correct answer to the left-hand one?
| Cosine | Verified | Question asked | Question the answer was stored for |
|---|---|---|---|
| 0.933 | no | How fast does light travel in a vacuum? | What is the speed of light in a vacuum? |
| 0.943 | no | What is the boiling point of water at sea level, in degrees Celsius? | At what temperature does water boil at sea level in Celsius? |
| 0.956 | no | Which city is the capital of Australia? | What is the capital of Australia? |
| 0.959 | no | What is the biggest planet in our solar system? | Which planet is the largest in the solar system? |
| 0.961 | no | At 25 °C, what pH does pure water have? | What is the pH of pure water at 25 degrees Celsius? |