DEV Community

ML Feature Store Versioning: Reproducible Quants

shakti tiwari — Wed, 12 Aug 2026 23:06:43 +0000

ML Feature Store Versioning: Reproducible Quants

By Shakti Tiwari · Educational only · Not investment advice

This article explains ml feature store versioning: reproducible quants from first principles. No live market numbers are quoted; the structure is what lasts.

Why this matters

ML Feature Store Versioning: Reproducible Quants is one of those subjects that sounds simple until you implement it, at which point the hidden complexity appears. The first version works on a laptop with a tiny file; the second version breaks at 3am when the WebSocket drops, the replay file is half-written, and you cannot tell which ticks you already stored. This article is a structural walkthrough: the concepts, the math where it helps, the code shape where it helps, and the failure modes that quietly cost money or correctness. No live market numbers are quoted because a number without a dated source is decoration, not education. The structure here does not expire, and unlike a specific price level, you can reuse it on the next dataset without re-deriving anything. If you only remember one sentence from this page, make it this: the boring parts are the product, and the interesting parts are a small fraction of what separates a demo from a system.

Core concept

At its heart, ml feature store versioning: reproducible quants is about being honest with your own assumptions. The trap is not that the idea is wrong; it is that a half-implemented version looks right in a demo and breaks in production. We separate the idea from the implementation so you can tell which one you actually have. A clean concept on paper can still produce a broken system if the boundary between 'what I meant' and 'what the code does' is never made explicit. Write the concept as a contract: given X observable at time t, the system produces Y, and any deviation is a bug, not a feature. A contract you can state in one sentence is also one you can test in one assertion, and that testability is the entire difference between an architecture and a wish. The cheaper you make the test, the more often you will run it, and the fewer surprises reach production.

The mechanism in three parts

The mechanism has three parts: what you observe, what you decide, and what you pay. Most tutorials show the first two and silently drop the third. The third is where real edges live or die. We model it explicitly: observation stamped at a bar, decision using only past data, cost taken from the realized fill not the midpoint. If you skip the stamping, you get look-ahead bias. If you skip the past-only rule, you get leakage. If you skip the realized cost, you get a strategy that looks profitable until a real order touches the book. Each omission is small in code and large in consequence. The reason tutorials drop the third part is that it is annoying to model: you need a realistic fill model, a fee schedule, and a tax rule, none of which appear in a clean formula. But the formula is the easy 10%; the fill model is the real 90%.

A structural example

Consider a minimal version. You collect state, compute a signal, act only if a risk budget allows, and log the outcome including the trades you discarded. The discipline is in the ordering: state from the past, signal from that state, cost from the real fill. Reorder any of those and the result becomes a story you tell yourself rather than a measurement. The example is deliberately boring because the point is the skeleton, not the flourish. A skeleton you can defend beats a polished curve built on a lie. When you later replace the minimal pieces with real infrastructure — a real feed, a real executor, a real risk engine — the skeleton stays; only the muscles change. That stability is the value of getting the order right once, up front, before any of the exciting parts are allowed to exist.

Idempotency and replay

Idempotency means running the same ingestion twice produces the same stored result, not duplicates. This matters the moment a connection resets and you replay a file: without a unique key per tick, you double-count volume, corrupt aggregates, and silently bias every feature built on top. The fix is mechanical: hash or sequence each record, upsert on conflict, never append blindly. Replay then becomes safe instead of dangerous, and recovery from a crash is a re-run, not an investigation. Idempotency is not a performance optimization; it is a correctness guarantee. Once you have it, you can stop fearing restarts, and you can stop writing the fragile 'resume from line N' logic that breaks the moment a partial line is written. The key is doing the dedupe at write time, where the data arrives, not at read time, where the damage is already done.

Where it breaks

It breaks at the seams: when the universe changes and you forgot to track it, when volatility regime shifts and your average hid the crash, when a parameter was tuned on the same data you tested on. Each break is a specific code or process error, not a mystery. Naming it is most of the fix. The pattern across all three is the same — a piece of state that should have been explicit was left implicit, and implicit state is where bugs hide. The reason these break at the seams and not in the middle is that the seam is the boundary between two systems you control at different times: the data you collected yesterday and the model you train today. If the boundary is not a first-class object in your code, it will be a first-class source of error in your P&L, and you will not see it until it is expensive.

How to make it robust

Robustness is boring: point-in-time data, walk-forward splits with a frozen holdout, explicit cost including tax on the correct leg, per-regime reporting, and a log of failed variants. None of these are optional if you intend to trust the number. They are the difference between a backtest and a bedtime story. Apply the same standard to the data layer: schema checked on write, duplicates rejected on key, checksums verified on replay. Robustness is not a feature you add at the end; it is a property you preserve at every step. Each shortcut you take 'just to ship' is a loan against a future debugging session, and the interest is paid in the currency you care about most — trust in your own numbers. The discipline is to treat the loan as real even when the deadline feels imaginary.

Common misconception

A common misconception about ml feature store versioning: reproducible quants is that more data or a fancier model fixes a broken loop. It does not. A leaky pipeline with ten years of data is still leaky; a deep model on mispriced fills still loses money. Fix the loop first; the model is the last thing you improve, not the first. The urge to reach for a bigger hammer is natural, because the hammer is visible and the loop is invisible. But the bottleneck is almost always upstream of the model, in the plumbing that decides what the model is allowed to see. Every hour spent on a better architecture while the ingestion is still leaking is an hour spent polishing the wrong thing. The honest move is unglamorous: close the leak, re-run, and only then ask whether the model needs to change.

Connection to the pipeline

This fits the companion stack: idempotent tick ingestion keeps the feature store reproducible, leakage-free XGBoost features keep the model honest, and a governed backtest keeps the loop from lying to you. The articles build on each other; read them in order if you are assembling a real system. Skipping the ingestion discipline to get faster to the model is the most expensive shortcut in quant, because every later step inherits the corruption. The pipeline is a chain, and a chain is only as strong as its weakest link; the ingestion link is also the one closest to the raw truth, so corrupting it poisons everything downstream while leaving the model code looking pristine. That is why the boring layer deserves the most scrutiny, not the least.

What good enough looks like

Good enough is not perfect; it is a system whose known limitations are written down. State the regimes you did not cover, the costs you approximated, the parameters you tuned. A result with honest scars beats a flawless one that hides them. That is the entire point of governed publishing: ship the analysis, keep the caveats attached, and let the reader see the seams instead of a polished surface. Good enough is also reproducible: someone else, with your notes, can rebuild the same number. If only you can reproduce it, it is not an analysis — it is a coincidence you happened to witness. Write the limitations as if the reader is a skeptic version of your future self, because that skeptic is exactly who will eventually read it.

A minimal checklist

Before you trust a pipeline built on this idea, answer: is each record uniquely keyed? Does replay produce identical state? Is observation stamped strictly in the past? Are costs modeled on the real fill? Are regimes reported separately? If any answer is no, the pipeline is a draft. The checklist is short because the failures are few and recurring; the cost of ignoring them is not. A checklist you cannot answer in under a minute is one you will skip under pressure, so keep it to five questions and make each one a yes-or-no. The goal is not completeness; it is a gate you can actually use at 3am when something broke and you need to know whether the data is trustworthy before you trust the alert it just fired.

The cost of skipping it

Skipping the discipline around ml feature store versioning: reproducible quants does not fail loudly. It fails as a slow drift: numbers that looked stable start disagreeing with the book, a replay produces a different P&L than the first run, and nobody can reproduce last month's report. By the time it is noticed, the corruption has propagated into every feature and every model trained on top. The recovery cost is then weeks, not minutes. Doing it right once is cheaper than explaining it forever. The drift is invisible because each individual discrepancy is small enough to blame on noise, and noise is always available as an excuse. The skill is to treat a small discrepancy as a signal, not a nuisance, because the small ones are how the large ones announce themselves quietly, weeks in advance.

A note on tooling

You do not need a heavy framework. A flat file with a unique key, an upsert on conflict, and a checksum per batch is enough for most retail-scale tick stores. The framework temptation is real, because frameworks feel like progress and a flat file feels like a stopgap. But the value is in the contract (unique key, past-only, verified replay), not in the library. A simple script that honors the contract beats a complex platform that violates it. When you outgrow the flat file, you will outgrow it in a way the contract already anticipated, and the migration will be a swap of storage, not a rewrite of assumptions. That is the whole point of respecting the constraint over chasing the tool.

Summary

ml feature store versioning: reproducible quants is not a feature you add; it is a constraint you respect. Stamp observation in the past, key every record, model the real cost, report by regime, and keep the discarded variants. Do those five and the system is defensible. Skip any and you have a story. The market does not care which one you believe; your P&L will. The constraint is tedious precisely because it is always correct and never exciting, and excitement is what your attention naturally follows. Training yourself to protect the tedious parts is the unglamorous core of building systems that survive contact with real data, real outages, and real money. Everything else in this article is a footnote to that one sentence.

Continue Reading

Shakti Tiwari writes about systematic options trading and ML. Follow on X · LinkedIn · GitHub · DEV. #ShaktiTiwariOnAI #NiftyOptions #QuantML #OptionsTrading #SystematicTrading

Extracting Recording Information From a Scanned Deed

Multigrid — Wed, 12 Aug 2026 23:06:27 +0000

The recording reference is how every other document in a title chain points at this one. It is a citation, not a database key, and the difference shows up the moment somebody tries to look one up using your normalised version of it.

What is on the stamp

The clerk’s stamp is applied at the moment of recording and carries more than a number. Typically it holds the county and state, the date and time of recording, a reference — book and page, an instrument number, or both — the recording officer’s title, and often the fees collected and any documentary transfer tax. Several of those fields matter to somebody downstream.

The time is the least obvious and among the most important. Recording statutes decide priority between competing interests, and under the race and race-notice regimes used by most states, priority turns on who recorded first. Two instruments recorded on the same day are separated only by the time on the stamp and the sequence in the recorder’s own numbering. An extraction that stores a date and discards a time has thrown away the tie-breaker.

The fee and tax figures are tempting as a route to sale price, because documentary transfer taxes are usually levied as a rate on consideration. Be careful: the rate is set by statute and varies by state and often by city, some conveyances are exempt and stamped as such, and some deeds recite a nominal consideration with the real price elsewhere. Extract the tax amount as an amount, not as a derived price, and leave the arithmetic to somebody who knows which rate applied on that date in that jurisdiction.

Book and page is not unique

The oldest convention is a physical one. Instruments were transcribed into bound volumes; the citation is the volume and the page it starts on. Different jurisdictions call the volume different things — Deed Book, Official Records Book, Volume, and in Maryland and parts of New York the older Liber and Folio — and some counties, having gone through microfilm, cite a reel and image or frame number instead.

The point that breaks pipelines is that these are separate series. A county may have run a Deed Book series, a Mortgage Book series and a Miscellaneous series at the same time, each with its own volume 1. So “Book 1245, Page 331” identifies a document only in combination with the county, the state and the series. A schema of two integers is not an identifier; it is two integers.

{
  "recording_reference_raw": "Recorded 04/09/2019 at 2:17 PM\nOFFICIAL RECORDS BK 1245 PG 331-334\nFRANKLIN COUNTY, OHIO",
  "jurisdiction": { "county": "Franklin", "state": "OH" },
  "series_as_written": "OFFICIAL RECORDS",
  "book": "1245",
  "page_start": "331",
  "page_end": "334",
  "recorded_date": "2019-04-09",
  "recorded_time": "14:17"
}

Note the page range. Long instruments span pages, and the citation may be written as a range, as a start page only, or as a start page plus a page count. All three occur, and a schema with a single integer page silently truncates the first.

Instrument numbers and the cutover

As recording offices moved to electronic systems, most adopted a sequential instrument number — also called a document number, file number or reception number — usually with a year prefix, such as 2019-0043221 or 201900043221. It is assigned in order of receipt, so it encodes the same priority information the time stamp does.

Every county made that change on its own date, and around that date the records are mixed. Documents recorded before the cutover have a book and page and no instrument number; documents after have an instrument number and, in many counties, a book and page as well, because the office kept assigning them for continuity. So the correct schema has both references as optional and requires at least one, not a single reference field with a type discriminator that has to be guessed — the general case of designing a schema for variants you have not seen.

The formatting traps are mundane and expensive. Leading zeros are significant: 2019-0043221 stored as the integer 43221 no longer matches the recorder’s search index. Hyphens appear or do not, sometimes within the same county. Prefixes are used for document class in some offices. And the year prefix is a two-digit year often enough that a naive parse of 19-0043221 as a year is wrong in both directions.

There is no national registry of these formats. Every recording office publishes its own conventions and its own search interface, and the only authority for what a reference in a given county looks like is that county’s recorder or clerk. Treat any per-county pattern in your pipeline as configuration to be verified against that office, never as something inferred from a sample of documents.

Keep the raw string

Everything above is an argument for the same design: the raw stamp text is the field of record and the normalised fields are derived from it, stored beside it, and allowed to be null.

There are three reasons this is not merely cautious. First, the stamp text is what a person quotes when they cite the instrument, and reassembling “Official Records Book 1245, Page 331” from parsed fields requires knowing the county’s house style, which you do not. Second, when the parse is wrong, the raw string is the only way to find out; a corrected parser can be re-run over stored raw text and cannot be re-run over discarded raw text. Third, stamps contain things your schema does not have fields for — a clerk’s initials, an auditor’s transfer endorsement, a plat reference — and those turn out to matter about once a year.

Stamps are also physically awkward. They are frequently rotated, because the clerk stamped the page sideways in the margin; they are frequently the darkest thing on a light scan, or the lightest thing on a dark one; and they overlap printed text, which is the problem discussed from the other side in extracting grantor and grantee names. Detect and deskew the stamp region separately from the body text rather than running one recognition pass over the whole page; the general handling is in the OCR pipeline page.

Re-recorded and corrective instruments

The case that most often produces a wrong answer with high confidence is a re-recorded instrument. When a deed is recorded with a defect — a missing notarial seal, a typo in the legal description — it is commonly re-recorded, sometimes with a legend across the first page reading “RE-RECORDED TO CORRECT THE LEGAL DESCRIPTION”. That document now bears two recording stamps, usually months apart, and both are real. Each carries its own date, so both go through the same date-field validation as any other extracted date.

A pipeline that extracts “the” recording reference from such a page will pick one, and which one it picks depends on stamp position rather than on meaning. The correct output is a list of recording events with an order, plus the re-recording legend if present. The earlier stamp is the original recording; the later one is when the corrected instrument entered the record. Which of them controls for any particular purpose is a question for a title examiner, and the extraction’s job is to make sure the examiner can see that there were two.

Related but distinct is a separate corrective deed or scrivener’s affidavit: a new instrument with its own recording reference that recites the reference of the deed it corrects. That recited reference is a fourth kind of string on the page — a citation to another document, not to this one — and merging it into the recording-information field is a common and confusing error. Extract references to other instruments into their own array with a relationship label.

Extracting the Legal Description From a Property Deed

Multigrid — Wed, 12 Aug 2026 23:06:11 +0000

A legal description is not a description of the property. It is the instrument by which the property is identified, and two documents that describe the same parcel in different words are two different legal descriptions. That is why the extraction target is a block of text, not a set of coordinates.

Three systems, one field

Almost every American legal description is written in one of three conventions, and knowing which one you are looking at tells you what the rest of the string has to contain.

Metes and bounds. A perimeter walked as a sequence of calls, each a bearing and a distance from a starting monument: “thence North 12 degrees 34 minutes 20 seconds East, 150.00 feet to an iron pin”. Bearings are almost always quadrant bearings — a north or south reference, an angle under 90 degrees, and an east or west direction — rather than azimuths. The description begins at a point of beginning, which is frequently reached from a separate point of commencement, and it must return to the point of beginning. A description that does not close is a description with a missing or garbled call.
Lot and block. The parcel is a numbered lot on a recorded subdivision plat: a lot, a block, the subdivision name, and — the part that is often lost — the plat’s own recording reference and county. “Lot 7, Block 3” without the plat book and page and the county names nothing; the same lot and block exist in every county in the state.
Public Land Survey System. Aliquot parts of a section: “the Northeast Quarter of the Southwest Quarter of Section 14, Township 3 North, Range 5 West”, followed by the principal meridian, which is the part naive extractions drop. A township is nominally six miles square and holds 36 sections numbered in a serpentine order, and because the survey has to reconcile a rectangular grid with a curved earth, sections along the north and west edges of a township are irregular. Those irregular pieces are described as numbered government lots, not as quarters, which is why a PLSS description sometimes contains a lot number and is still not lot-and-block.

A single deed can carry more than one. A parcel assembled from an old farm and an adjoining platted lot will be described in PLSS for the first tract and lot-and-block for the second, joined by “AND” or numbered “Tract I / Tract II”. A schema with one string field per deed loses the second tract; a schema with an array of tracts does not.

The verbatim block is the deliverable

The temptation is to parse the calls into numbers on the way in: a list of bearings and distances, or a polygon. Resist it as the primary output. The legal description is operative text. If it is later quoted in a title commitment, a corrective deed or a complaint, the string that gets quoted has to be the string that is on the recorded page, punctuation and all. A pipeline that stores a reconstruction — even a faithful one — has stored a paraphrase of a legal instrument.

So the first extracted field is the description as written, preserved with its line breaks, its capitalisation and its abbreviations, and stored alongside the page image and the character offsets it came from. That gives a reviewer something to compare against, which is the only meaningful check available: there is no checksum on a metes and bounds call, no authority to look it up in, and no way to tell a correct bearing from a plausible wrong one without reading the page.

{
  "description_verbatim": "BEGINNING at an iron pin at the Southeast corner ...",
  "description_system": "metes_and_bounds",
  "source_page_range": [3, 4],
  "source_bbox_by_page": [[72, 410, 540, 720], [72, 96, 540, 210]],
  "derived": {
    "calls": [
      { "bearing": "N 12-34-20 E", "distance_ft": 150.0, "monument": "iron pin" }
    ],
    "derivation": "model-parsed, not authoritative"
  }
}

What a parsed description is actually for

The parsed form is still worth producing, but for a different job than the verbatim block. Its value is that it fails. A call list that does not close back to the point of beginning is evidence of a transcription error, and it is one of the very few automatic checks available on this document. Sum the calls as vectors; if the closure error is large relative to the perimeter, something in the string is wrong — a digit dropped from a distance, a bearing whose quadrant letter was misread, a whole call lost at a page break.

Treat that as a flag, not a correction. Real recorded descriptions fail to close for reasons that have nothing to do with your OCR: old surveys, curve calls described by arc length and radius that a naive summation treats as straight lines, and calls that run “along the centerline of the creek” where the boundary is a natural monument with no bearing at all. Surveyors resolve conflicts between calls by a documented order of priority in which natural monuments outrank artificial monuments, which outrank courses and distances, which outrank stated acreage. That ordering is a surveying and legal doctrine, not an extraction rule, and a pipeline that starts applying it has stopped extracting and started opining.

Where transcription breaks

The failures here are unusually concentrated in a few characters, all of which appear in every metes and bounds description and nowhere else in the document.

The degree sign, read as a zero, a lowercase o, a superscript or nothing at all. “N 12 34 20 E” with the symbols gone is unparseable and, worse, sometimes parses wrongly.
Minute and second marks — a prime and a double prime — read as an apostrophe and a quotation mark, then normalised into curly quotes by a downstream text cleaner. That normalisation is invisible in a diff viewer and destroys the field.
Feet expressed with the same prime character, so 150.00' and 150 minutes are one glyph apart in context.
Fraction glyphs in PLSS descriptions: a one-quarter character, the digits 1/4, and the spelled word “Quarter” all occur, and typewritten deeds sometimes use NE1/4SW1/4 with no separators at all.
Descriptions that continue onto a second page or onto an attached “Exhibit A”, which is frequently a different scan quality, a different typeface and occasionally a different orientation. The exhibit, not the body, is usually the operative text where both exist.

Most of these are properties of the scan rather than of the model, so they are addressed upstream: see the OCR pipeline page for the recognition side and PDF parsing for how text layers and image layers disagree on documents like these.

The clause everyone drops

The single most consequential omission in this field is an exception. A description that ends “LESS AND EXCEPT that portion conveyed to the County for road right-of-way by instrument recorded in Book 1102, Page 88” describes a smaller parcel than the same description without those words. So does “SAVE AND EXCEPT”, “EXCEPTING THEREFROM”, and a reservation of mineral rights. Conversely “TOGETHER WITH” adds something — typically an appurtenant easement — and is equally load bearing.

These clauses sit after the closing call, which is exactly where a model that has been told to extract “the legal description” decides it has finished. Give the extraction an explicit end condition instead: the description runs to the end of the descriptive text, including every exception, reservation and appurtenance, and stops at the next operative clause of the deed — the habendum, the warranty covenants, or the “subject to” list. Then add a separate boolean for whether any exception language was found, so that a missing exception on a parcel that has one shows up as a disagreement between two fields rather than as silence. Pair it with a per-field confidence signal and route anything ambiguous to a human who can look at the page.

Volatility Surface Basics: Term Structure Explained

shakti tiwari — Wed, 12 Aug 2026 23:06:05 +0000

Volatility Surface Basics: Term Structure Explained

By Shakti Tiwari · Educational only · Not investment advice

This article explains volatility surface basics: term structure explained from first principles. No live market numbers are quoted; the structure is what lasts.

Why this matters

Volatility Surface Basics: Term Structure Explained is one of those subjects that sounds simple until you implement it, at which point the hidden complexity appears. The first version works on a laptop with a tiny file; the second version breaks at 3am when the WebSocket drops, the replay file is half-written, and you cannot tell which ticks you already stored. This article is a structural walkthrough: the concepts, the math where it helps, the code shape where it helps, and the failure modes that quietly cost money or correctness. No live market numbers are quoted because a number without a dated source is decoration, not education. The structure here does not expire, and unlike a specific price level, you can reuse it on the next dataset without re-deriving anything. If you only remember one sentence from this page, make it this: the boring parts are the product, and the interesting parts are a small fraction of what separates a demo from a system.

Core concept

At its heart, volatility surface basics: term structure explained is about being honest with your own assumptions. The trap is not that the idea is wrong; it is that a half-implemented version looks right in a demo and breaks in production. We separate the idea from the implementation so you can tell which one you actually have. A clean concept on paper can still produce a broken system if the boundary between 'what I meant' and 'what the code does' is never made explicit. Write the concept as a contract: given X observable at time t, the system produces Y, and any deviation is a bug, not a feature. A contract you can state in one sentence is also one you can test in one assertion, and that testability is the entire difference between an architecture and a wish. The cheaper you make the test, the more often you will run it, and the fewer surprises reach production.

The mechanism in three parts

A structural example

Idempotency and replay

Where it breaks

How to make it robust

Common misconception

A common misconception about volatility surface basics: term structure explained is that more data or a fancier model fixes a broken loop. It does not. A leaky pipeline with ten years of data is still leaky; a deep model on mispriced fills still loses money. Fix the loop first; the model is the last thing you improve, not the first. The urge to reach for a bigger hammer is natural, because the hammer is visible and the loop is invisible. But the bottleneck is almost always upstream of the model, in the plumbing that decides what the model is allowed to see. Every hour spent on a better architecture while the ingestion is still leaking is an hour spent polishing the wrong thing. The honest move is unglamorous: close the leak, re-run, and only then ask whether the model needs to change.

Connection to the pipeline

What good enough looks like

A minimal checklist

The cost of skipping it

Skipping the discipline around volatility surface basics: term structure explained does not fail loudly. It fails as a slow drift: numbers that looked stable start disagreeing with the book, a replay produces a different P&L than the first run, and nobody can reproduce last month's report. By the time it is noticed, the corruption has propagated into every feature and every model trained on top. The recovery cost is then weeks, not minutes. Doing it right once is cheaper than explaining it forever. The drift is invisible because each individual discrepancy is small enough to blame on noise, and noise is always available as an excuse. The skill is to treat a small discrepancy as a signal, not a nuisance, because the small ones are how the large ones announce themselves quietly, weeks in advance.

A note on tooling

Summary

volatility surface basics: term structure explained is not a feature you add; it is a constraint you respect. Stamp observation in the past, key every record, model the real cost, report by regime, and keep the discarded variants. Do those five and the system is defensible. Skip any and you have a story. The market does not care which one you believe; your P&L will. The constraint is tedious precisely because it is always correct and never exciting, and excitement is what your attention naturally follows. Training yourself to protect the tedious parts is the unglamorous core of building systems that survive contact with real data, real outages, and real money. Everything else in this article is a footnote to that one sentence.

Continue Reading

Shakti Tiwari writes about systematic options trading and ML. Follow on X · LinkedIn · GitHub · DEV. #ShaktiTiwariOnAI #NiftyOptions #QuantML #OptionsTrading #SystematicTrading

Extracting Grantor and Grantee Names From a Deed

Multigrid — Wed, 12 Aug 2026 23:05:56 +0000

Grantor and grantee are not two name fields. They are one relationship with a direction, and the direction is what the county recorder indexes on. Get it backwards and every chain of title built on your data runs the wrong way.

Direction is the whole field

The grantor conveys; the grantee receives. Recorders have maintained two indexes on that distinction for as long as there have been recording offices: a grantor index, usually called the direct index, and a grantee index, usually called the indirect or reverse index. Title work runs backwards through the grantee index — find the instrument in which the current owner was the grantee, note who the grantor was, then find the instrument in which that party was the grantee, and so on back through the chain.

This is why a swapped pair is worse than a missing pair. A missing pair is a gap somebody notices. A swapped pair produces a chain that links up cleanly and describes a sequence of conveyances that never happened. Nothing downstream can detect it, because both names are real and both appear on the document.

The same relationship appears under other names on other instruments and the roles do not map one-to-one. A mortgage has a mortgagor (the borrower, who grants the security interest) and a mortgagee. A deed of trust has three parties: a trustor or grantor, a trustee who holds bare legal title, and a beneficiary who is the lender. If your schema has only grantor and grantee, a deed of trust does not fit in it, and forcing the beneficiary into the grantee slot loses the trustee entirely. Type the instrument first, then extract the party roles that instrument type actually has.

Anchor on the granting clause

Deeds are formulaic, and the formula is what to anchor on. The granting clause is a single sentence built around an operative verb phrase, and the parties sit on either side of it:

THIS INDENTURE, made this 3rd day of April, 2019, between
ALEX RIVERA and JORDAN RIVERA, husband and wife, of Franklin County,
party of the first part, and
MERIDIAN HOLDINGS LLC, a Delaware limited liability company,
party of the second part;

WITNESSETH, that the said party of the first part, for and in
consideration of Ten Dollars ($10.00) and other good and valuable
consideration, does hereby GRANT, BARGAIN, SELL AND CONVEY unto the
said party of the second part, its successors and assigns forever,
the following described real estate ...

The party before the verb grants; the party after “unto” receives. That holds across the vocabulary variants: a warranty deed says “grant, bargain, sell and convey”, a quitclaim says “remise, release and forever quitclaim”, a special warranty deed narrows the covenant but keeps the same shape. Older instruments use “party of the first part” and “party of the second part” as a layer of indirection, and the names are defined in a preamble above; a model that reads only the granting clause of such a deed will extract two phrases rather than two names, which is at least an honest failure.

Do not anchor on position on the page. The block at the top left is often “Prepared by” and the block beside it “When recorded return to” — a law firm and a title company respectively, and neither is a party. Extractions that key on “the first name on the document” return the drafting attorney with high confidence, which is the worst combination of properties an extracted field can have. Instructing the model to quote the span it used, as in extraction prompts, makes that failure visible in review.

Names carry capacity

The party field is rarely just a name. What follows it is frequently part of the identification and cannot be dropped:

Vesting language for the grantee. “as joint tenants with right of survivorship”, “as tenants in common”, “as tenants by the entirety”, “a married person as sole and separate property”. This determines what happens on death and is a distinct field, not a suffix on the name string.
Representative capacity. “PAT MORGAN, as Trustee of the Morgan Family Revocable Trust dated 12 June 2014” is the trust taking title, not the individual. Store the trust name and the trust date as their own fields; the trust date distinguishes two trusts with the same family name. The same applies to “as Personal Representative of the Estate of” and “as Attorney-in-Fact for”.
Entity form. “a Delaware limited liability company” is jurisdictional information that matters when two entities share a name across states.
Latin abbreviations in older deeds. et ux. means “and wife”, et vir “and husband”, et al. “and others”. These are not names and they are not noise: they signal a party who exists and is not printed. Extracting “ALEX RIVERA et ux.” as a complete grantor list is wrong, and the correct output records that an unnamed additional grantor is present.

When the stamp lands on the name

The recording stamp is applied by the clerk after execution, so it physically sits on top of whatever was already printed there. On a deed whose margins are narrow, that is frequently the first line or two of the preamble — which is where the grantor’s name is. Scanned from microfilm, the result is a name with a rectangle of purple ink through it, or a name with two characters replaced by a county seal.

The failure mode to design against is not the model refusing. It is the model completing. A language model asked for a grantor name from a page reading “ALE_ RI_ERA” will very often return “ALEX RIVERA” — a plausible completion of a common name, indistinguishable in the output from a clean read. Names are precisely the field where inference is illegitimate: unlike a date or a total, a name has no arithmetic and no context that constrains it, and a plausible wrong name is a real person.

So the response is structural. Ask for the characters actually legible and a separate occlusion flag, rather than for a name:

{
  "role": "grantor",
  "name_as_written": "ALE? RI?ERA",
  "legible": false,
  "occlusion": "recording stamp overlaps characters 4 and 7",
  "page": 1,
  "bbox": [96, 132, 388, 156]
}

A field that a model cannot read should arrive as an unread field with a reason, never as a guess. The point of the bounding box is that a reviewer opens the page at the right place instead of hunting for it.

The acknowledgment is a second reading

There is one genuinely useful redundancy on a deed, and it is underused. Every recorded deed carries a notarial acknowledgment near the signatures, and the acknowledgment restates the grantor’s name in full: “On this 3rd day of April, 2019, before me personally appeared ALEX RIVERA and JORDAN RIVERA, known to me to be the persons described in and who executed the foregoing instrument”. It is a second, physically separate rendering of the same name, at the bottom of the page rather than the top, and therefore almost never occluded by the same stamp.

Extract it as its own field and compare. Agreement raises confidence on both. Disagreement is informative in itself — it is frequently a middle initial or a suffix present in one place and not the other, occasionally a genuinely different party where a deed was signed by an attorney-in-fact, and sometimes a scrivener’s error that is exactly what a title examiner is looking for. Note that the acknowledgment covers only the signing parties, so it corroborates the grantor and says nothing about the grantee, who does not sign an ordinary deed at all. The grantee therefore has no second source and stays a single-read field.

Party names on identity-adjacent documents are one of the places where providers’ safety behaviour differs: the same page that one vision model transcribes will occasionally draw a refusal from another, and a refusal in the middle of a county-scale backlog is an operational problem rather than a content one. If you route through a gateway like Multigrid, the fallback is a routing rule rather than a second integration — one key, one request shape, and the retry on a different provider shows up in the same per-request cost log as the original.

The recording stamp is also the source of the citation for the instrument itself, which is a separate extraction with separate failure modes — see extracting recording information, and the legal description page for the other half of what makes a deed a deed.

Deduplicating Flaky Failures Across a Large Prompt Test Suite

Multigrid — Wed, 12 Aug 2026 23:05:40 +0000

When a suite that calls a model goes red, it usually goes red in clusters: one rate limit, one bad deploy, one model alias moving, and forty tests fail at once. Reading that as forty problems is the default behaviour of every test report, and it is wrong in a way that costs hours.

Why grouping by test name fails

A test report is organised by the thing the runner knows: file, class, test name. That organisation assumes failures are independent, which is a decent assumption for unit tests and a bad one for tests that all go through the same HTTP client to the same provider. When the shared dependency wobbles, every test that touches it fails, and the report presents them as unrelated because their names are unrelated.

The consequence is not just wasted reading. It distorts every downstream number: your flake counts spike for forty tests that are individually fine, your triage backlog fills with duplicates, and a genuine single-test regression that happened during the same run is buried in the noise. Grouping by cause is what makes the run legible — one incident, one row, forty affected tests, and any test that failed for a different reason standing out immediately.

What goes into a signature

A signature is a short string computed from a failure such that two failures with the same root cause produce the same string, and two with different causes do not. The fields worth including, in priority order:

The HTTP status, if there was one. A 429 and a 500 are different incidents even if the resulting exception text is identical. Capture this in the test harness — it is not in the traceback.
The provider error type or code. rate_limit_exceeded, overloaded_error, context_length_exceeded and their equivalents are the most discriminating field available, and they are stable strings.
The exception class name. AssertionError versus ValidationError versus a transport timeout is the first split most reports never make.
The assertion site. The last stack frame inside your own code — file and function, not line number, because line numbers move with every edit and would fragment the group across commits.
The served model id. The same assertion failing on two different models is two problems.

What must not go in: the test name, the full message, the request id, or any timestamp. Include the test name and every group has exactly one member, which is where you started.

The trade-off to keep in view is that a signature can be too coarse as easily as too fine. Hashing only the exception class collapses every AssertionError in the suite into one group, which is a single useless row. The test of a good signature is behavioural rather than aesthetic: when you look at the largest group, every member should have the same fix. If two members would be fixed by different people, add a field; if two groups would be fixed by the same change, remove one.

The normalisation rules

The message is where the useful discrimination lives and also where all the entropy is, so it needs normalising before it is hashed. Six substitutions do most of the work, and the order matters — do the specific ones before the general ones.

import re

SUBS = [
    (re.compile(r"\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-"
                r"[0-9a-f]{4}-[0-9a-f]{12}\b", re.I), "<uuid>"),
    (re.compile(r"\b(req|chatcmpl|msg)_[A-Za-z0-9]{6,}\b"), "<id>"),
    (re.compile(r"\b\d{4}-\d{2}-\d{2}[T ][\d:.]+Z?\b"), "<ts>"),
    (re.compile(r"0x[0-9a-f]+", re.I), "<addr>"),
    (re.compile(r"'[^']{40,}'"), "'<long>'"),   # inlined prompts and completions
    (re.compile(r"\b\d+\b"), "<n>"),          # do this LAST
]

def normalise(message: str) -> str:
    text = message.strip().split("\n")[0][:400]
    for pattern, replacement in SUBS:
        text = pattern.sub(replacement, text)
    return text

Two of those deserve a warning. Replacing every integer with <n> collapses “expected 1 tool call, got 0” and “expected 1 tool call, got 3” into one group, which is usually what you want and occasionally hides a real distinction; run it last so the more specific patterns claim their matches first. And truncating to the first line drops the model’s output from the message, which is the entire point — a completion embedded in an assertion message is unique every time and will defeat any grouping if you leave it in.

A script over your JUnit XML

Every runner worth using emits JUnit XML: pytest with --junitxml, Vitest with --reporter=junit --outputFile, Playwright with its junit reporter. That file is the input, and the standard library can parse it — no dependency required.

Make every job emit JUnit XML to a known path and upload it as an artifact. Do this even for jobs that pass; the pass rows are the denominator of every rate you will later want.
Parse each <testcase> and read its <failure> or <error> child. Where your runner emits the rerun dialect, also read <flakyFailure> and <rerunFailure>: those are the attempts that were retried, and they are the flake signal.
Compute the signature and group. Print the groups largest first.

import hashlib, sys, xml.etree.ElementTree as ET
from collections import defaultdict

FAILURE_TAGS = ("failure", "error", "flakyFailure", "rerunFailure")

def signature(case, node):
    parts = [
        node.tag,
        node.get("type") or "",
        normalise(node.get("message") or ""),
        case.get("classname") or "",
    ]
    raw = "|".join(parts)
    return hashlib.sha1(raw.encode()).hexdigest()[:10], raw

groups = defaultdict(list)
for path in sys.argv[1:]:
    for case in ET.parse(path).iter("testcase"):
        for node in case:
            if node.tag in FAILURE_TAGS:
                key, raw = signature(case, node)
                groups[key].append((case.get("name"), raw))

for key, members in sorted(groups.items(), key=lambda kv: -len(kv[1])):
    print(f"{len(members):4d}  {key}  {members[0][1][:110]}")
    for name, _ in members[:5]:
        print(f"        {name}")
    if len(members) > 5:
        print(f"        ... and {len(members) - 5} more")

The script deliberately reads several files at once, because the unit you want to group over is a CI run, not a job. A suite split across eight parallel shards writes eight XML files, and a rate limit that hit all eight is one incident; grouping per file would report it as eight. Pass the whole artifact directory.

Note that classname is in the signature but the test name is not. Including the class keeps failures in genuinely unrelated modules apart while still collapsing the twenty tests inside one module; drop it entirely if your suite groups everything into one class, since it then contributes nothing.

Using the groups

The output changes what a red build asks of you. A single group with forty members and a rate_limit_exceeded code is one action: reduce the suite’s concurrency, which the locally-green, CI-red page covers. A group of one, with an AssertionError at a specific function in your own code, is the thing you actually have to read.

Persist the signature alongside every attempt rather than computing it only in the moment. Once it is a column, the questions that were hard become trivial: which signature is new this week, which one accounts for most of the suite’s red time, whether the group that spiked on Tuesday has appeared before. That is one more field in the table a flake dashboard already needs, and it is the field that makes the dashboard worth building.

The rerun-related JUnit elements are a Surefire-derived extension, not part of any single standard, and support varies by runner and by CI product. Check what your runner actually emits before relying on <flakyFailure> being present.

Theta Harvesting: Collecting Decay Without Getting Burned

shakti tiwari — Wed, 12 Aug 2026 23:05:26 +0000

Theta Harvesting: Collecting Decay Without Getting Burned

By Shakti Tiwari · Educational only · Not investment advice

This article explains theta harvesting: collecting decay without getting burned from first principles. No live market numbers are quoted; the structure is what lasts.

Why this matters

Theta Harvesting: Collecting Decay Without Getting Burned is one of those subjects that sounds simple until you implement it, at which point the hidden complexity appears. The first version works on a laptop with a tiny file; the second version breaks at 3am when the WebSocket drops, the replay file is half-written, and you cannot tell which ticks you already stored. This article is a structural walkthrough: the concepts, the math where it helps, the code shape where it helps, and the failure modes that quietly cost money or correctness. No live market numbers are quoted because a number without a dated source is decoration, not education. The structure here does not expire, and unlike a specific price level, you can reuse it on the next dataset without re-deriving anything. If you only remember one sentence from this page, make it this: the boring parts are the product, and the interesting parts are a small fraction of what separates a demo from a system.

Core concept

At its heart, theta harvesting: collecting decay without getting burned is about being honest with your own assumptions. The trap is not that the idea is wrong; it is that a half-implemented version looks right in a demo and breaks in production. We separate the idea from the implementation so you can tell which one you actually have. A clean concept on paper can still produce a broken system if the boundary between 'what I meant' and 'what the code does' is never made explicit. Write the concept as a contract: given X observable at time t, the system produces Y, and any deviation is a bug, not a feature. A contract you can state in one sentence is also one you can test in one assertion, and that testability is the entire difference between an architecture and a wish. The cheaper you make the test, the more often you will run it, and the fewer surprises reach production.

The mechanism in three parts

A structural example

Idempotency and replay

Where it breaks

How to make it robust

Common misconception

A common misconception about theta harvesting: collecting decay without getting burned is that more data or a fancier model fixes a broken loop. It does not. A leaky pipeline with ten years of data is still leaky; a deep model on mispriced fills still loses money. Fix the loop first; the model is the last thing you improve, not the first. The urge to reach for a bigger hammer is natural, because the hammer is visible and the loop is invisible. But the bottleneck is almost always upstream of the model, in the plumbing that decides what the model is allowed to see. Every hour spent on a better architecture while the ingestion is still leaking is an hour spent polishing the wrong thing. The honest move is unglamorous: close the leak, re-run, and only then ask whether the model needs to change.

Connection to the pipeline

What good enough looks like

A minimal checklist

The cost of skipping it

Skipping the discipline around theta harvesting: collecting decay without getting burned does not fail loudly. It fails as a slow drift: numbers that looked stable start disagreeing with the book, a replay produces a different P&L than the first run, and nobody can reproduce last month's report. By the time it is noticed, the corruption has propagated into every feature and every model trained on top. The recovery cost is then weeks, not minutes. Doing it right once is cheaper than explaining it forever. The drift is invisible because each individual discrepancy is small enough to blame on noise, and noise is always available as an excuse. The skill is to treat a small discrepancy as a signal, not a nuisance, because the small ones are how the large ones announce themselves quietly, weeks in advance.

A note on tooling

Summary

theta harvesting: collecting decay without getting burned is not a feature you add; it is a constraint you respect. Stamp observation in the past, key every record, model the real cost, report by regime, and keep the discarded variants. Do those five and the system is defensible. Skip any and you have a story. The market does not care which one you believe; your P&L will. The constraint is tedious precisely because it is always correct and never exciting, and excitement is what your attention naturally follows. Training yourself to protect the tedious parts is the unglamorous core of building systems that survive contact with real data, real outages, and real money. Everything else in this article is a footnote to that one sentence.

Continue Reading

Shakti Tiwari writes about systematic options trading and ML. Follow on X · LinkedIn · GitHub · DEV. #ShaktiTiwariOnAI #NiftyOptions #QuantML #OptionsTrading #SystematicTrading

Getting the Decimal Comma or Point Right by Locale in AI Output

Multigrid — Wed, 12 Aug 2026 23:05:24 +0000

A model writing German produces 1,234,56. It has correctly learned that German uses a decimal comma and has not learned that German therefore uses something else for the thousands, so it applied one rule and left the other alone. The result is not a foreign-looking number. It is not a number.

The half-right bug

The two separators are a matched pair. A locale that uses a comma to mark the decimal cannot also use a comma to group thousands, because the string would be unparseable — which is exactly what happens when a generator applies only one half of the pair.

It happens because the decimal comma is the salient, frequently-stated fact (“Germans use a comma for decimals”) while the thousands separator is the consequence nobody writes down. A model following an instruction to use European number format has strong evidence for the first and weak evidence for the second, and applies what it has.

Three variants of the resulting output are worth recognising:

Both commas: 1,234,56. Unparseable. This is the most common and, mercifully, the most visible.
Decimal swapped, grouping dropped: 1234,56. Correct as a number and unidiomatic above four digits; harmless in small amounts, noticeably wrong in a financial table.
Fully inverted but inconsistently applied: 1.234,56 in one paragraph and 1,234.56 two paragraphs later, in the same document. This is the worst one, because each individual number is well-formed and only the document as a whole is wrong.

The conventions, worked

One thousand two hundred and thirty-four and fifty-six hundredths:

en-US   1,234.56    group comma,      decimal point
en-GB   1,234.56    group comma,      decimal point
de-DE   1.234,56    group point,      decimal comma
es-ES   1.234,56    group point,      decimal comma
it-IT   1.234,56    group point,      decimal comma
pt-BR   1.234,56    group point,      decimal comma
fr-FR   1 234,56    group narrow no-break space, decimal comma
ru-RU   1 234,56    group no-break space, decimal comma
pl-PL   1 234,56    group no-break space, decimal comma
sv-SE   1 234,56    group no-break space, decimal comma
de-CH   1'234.56    group apostrophe, decimal point
en-IN   1,234.56    group comma but Indian grouping above five digits

The French, Russian, Polish and Swedish rows are the ones that break code, and the reason is that the separator is not a plain space. CLDR uses a no-break space so the number cannot be split across a line, and for French the group separator is U+202F NARROW NO-BREAK SPACE specifically. Any string comparison, regular expression or trimming step written against U+0020 will not match it, and any test written by typing a space into an editor will fail against correct output.

The Swiss row is a third distinct pattern that undermines any two-camp model of the world: a German-speaking locale with a decimal point and an apostrophe for grouping. And the Indian row points at the separate problem that grouping is not always in threes, which is the lakh-crore system.

What the standards bodies say

There is a formal recommendation, and it is worth knowing because it appears in scientific and technical documents and looks like an error to people who have not met it. The International Bureau of Weights and Measures, in the SI Brochure, states that either a comma or a point may be used as the decimal marker, and that digits should be separated into groups of three by a thin space and never by a comma or a point — precisely so that the grouping character cannot be confused with the decimal marker in either convention.

That is why a physics paper writes 1 234.56 where a newspaper would write 1,234.56. If you are generating technical or scientific copy, the space-grouped form is the correct one and a reviewer who removes the spaces is introducing an error. If you are generating anything else, follow CLDR’s locale data instead.

Parsing is where it becomes dangerous

Formatting wrong is embarrassing. Parsing wrong loses money. Consider a model extracting an amount from a European invoice and handing the string to code that calls a naive numeric parser:

parseFloat("1.234,56")   // 1.234   — stops at the comma
Number("1.234,56")       // NaN
parseFloat("1.234")      // 1.234   — was one thousand two hundred and thirty-four

The third line is the one that gets into production. 1.234 is a perfectly valid float, so nothing throws, nothing warns, and an amount of 1,234 euro is recorded as 1.234 euro — off by a factor of a thousand, in a field that no validation will question because it is a plausible number. The same string in the other direction turns 1.234 into 1234.

The defence is to never let a locale-formatted numeric string reach a parser. Have the extraction produce a normalised value with an explicit, single convention — a JSON number, or a string in 1234.56 form with no grouping at all — and have the model report the source formatting separately if you need it. Where you genuinely must parse a localised string, use a locale-aware parser and pass the locale explicitly; do not infer it from the string, because 1.234 is ambiguous and always will be.

The interchange format where this bites hardest is CSV, and the reason is worth spelling out because it produces a file that opens correctly on one desk and not on another. A locale that uses a decimal comma cannot use a comma to separate CSV fields, so the convention in those locales is a semicolon — and spreadsheet software follows the operating system list separator when it decides how to split a .csv file. The same file therefore parses into columns on a US machine and into a single column on a German one, or worse, splits a European number across two columns. If you are generating tabular exports, either emit a real spreadsheet format, or emit unambiguous machine-format numbers with a comma delimiter and quote every field, and state the convention in a header row rather than leaving the receiving application to guess.

The fix

Instruct the model to emit numbers with a decimal point and no grouping separator at all: 1234.56. This is a machine format, not a display format, and it should never reach a reader.
Validate that instruction with a pattern — ^-?\d+(\.\d+)?$ — rather than trusting it. A number with a comma in it is a rejected response, not a value to clean up.
Format at the point of display with an ICU-backed formatter and an explicit locale, so the grouping and decimal characters come from CLDR data rather than from a rule anyone remembered.
Test with a value above 999 and below 10,000. Anything smaller has no grouping separator and will pass under any convention, which is how this bug reaches production.

new Intl.NumberFormat("de-DE").format(1234.56)   // "1.234,56"
new Intl.NumberFormat("fr-FR").format(1234.56)   // "1 234,56"  (U+202F)
new Intl.NumberFormat("de-CH").format(1234.56)   // "1'234.56"

When the currency symbol is involved as well, its position and spacing are a separate piece of locale data and are covered in where the currency symbol goes. They resolve from the same formatter call, which is the argument for making that call once rather than assembling a monetary string by concatenation.

Riverpod to BlocSignal: Incremental Migration and Zero-Codegen Signals for Flutter

Randal L. Schwartz — Wed, 12 Aug 2026 23:05:09 +0000

Introduction: Combining Riverpod Safety with Zero Codegen

If you've been building Flutter apps with Riverpod, you already know the joy of compile-time provider safety, auto-disposal, and synchronous notification propagation via ProviderListenable. Riverpod fundamentally raised the bar for Flutter state management.

However, as projects grow, many Riverpod developers run into familiar friction points:

Code Generation Overhead: Depending heavily on riverpod_generator and waiting on build_runner watches during rapid UI iteration.
Complex Provider Trees: Managing nested ProviderScope overrides and .family cache eviction policies when scaling large teams.
All-or-Nothing Migration Concerns: Wanting to trial new reactive primitives or event-driven BLoC architectures without rewriting an entire codebase.

What if you could combine the compile-time safety and synchronous reactivity you love in Riverpod with zero code generation, fine-grained signal graph reactivity, and the ability to trial or migrate incrementally screen-by-screen?

Enter BlocSignal and bloc_signals_riverpod.

The Core Ergonomics: Riverpod `Notifier` vs. `CubitSignal`

Let's compare the classic Todos application—one of the benchmark examples in the official Riverpod monorepo—ported directly to BlocSignal.

The Riverpod Approach (Requires `@riverpod` & `build_runner`)

In modern Riverpod, creating a todo list with reactive filtering typically involves a generated Notifier and separate provider getters or ref.watch selectors:

@riverpod
class TodoList extends _$TodoList {
  @override
  List<Todo> build() => const [];

  void addTodo(String description) {
    state = [...state, Todo(id: DateTime.now().toString(), description: description)];
  }

  void toggle(String id) {
    state = [
      for (final todo in state)
        if (todo.id == id) todo.copyWith(completed: !todo.completed) else todo
    ];
  }
}

// Derived filter provider
@riverpod
List<Todo> filteredTodos(Ref ref) {
  final todos = ref.watch(todoListProvider);
  final filter = ref.watch(todoFilterProvider);
  return switch (filter) {
    TodoFilter.all => todos,
    TodoFilter.active => todos.where((t) => !t.completed).toList(),
    TodoFilter.completed => todos.where((t) => t.completed).toList(),
  };
}

The BlocSignal Approach (100% Pure Dart 3, No Codegen)

With BlocSignal, your state container is a standard handwritten Dart class (CubitSignal<List<Todo>>). Reactive derivations like filteredTodos and uncompletedCount are declared inline using signals computed():

import 'package:bloc_signals/bloc_signals.dart';
import 'package:flutter/foundation.dart';
import 'package:signals_core/signals_core.dart';

class TodosCubit extends CubitSignal<List<Todo>> {
  // Pass `equals: listEquals` to enforce value-based list equality for state de-duplication!
  TodosCubit([List<Todo> initialTodos = const []])
      : super(initialState: initialTodos, equals: listEquals) {
    // 1. Reactive filter signal
    filter = signal(TodoFilter.all);

    // 2. Synchronously derived computed signals
    filteredTodos = computed(() {
      return switch (filter.value) {
        TodoFilter.all => stateValue,
        TodoFilter.active => stateValue.where((t) => !t.completed).toList(),
        TodoFilter.completed => stateValue.where((t) => t.completed).toList(),
      };
    });

    uncompletedCount = computed(() => stateValue.where((t) => !t.completed).length);
  }

  late final Signal<TodoFilter> filter;
  late final ReadonlySignal<List<Todo>> filteredTodos;
  late final ReadonlySignal<int> uncompletedCount;

  void addTodo(String description) {
    emit([...stateValue, Todo(id: DateTime.now().toString(), description: description)]);
  }

  void toggle(String id) {
    emit([
      for (final todo in stateValue)
        if (todo.id == id) todo.copyWith(completed: !todo.completed) else todo
    ]);
  }

  void setFilter(TodoFilter newFilter) => filter.value = newFilter;

  @override
  Future<void> close() async {
    filter.dispose();
    filteredTodos.dispose();
    uncompletedCount.dispose();
    await super.close();
  }
}

What Changed?

No build_runner: No .g.dart generated files, no background watchers, no build step delays.
Built-In Custom Equality (equals: listEquals): Because Dart List instances don't override == by default, passing equals: listEquals configures the underlying signal graph to de-duplicate state emissions based on list content equality.
Fine-Grained Signal Graph: Updates to filter.value or calling emit(...) re-evaluate computed() derivations synchronously and notify only dependent widgets.
Explicit Container Lifecycles: Disposing the cubit disposes its internal signals automatically.

Zero Risk: Incremental Trial & Bidirectional Interop

You do not need to rewrite your application to try BlocSignal. With the bloc_signals_riverpod package, you can seamlessly bridge the two frameworks in both directions.

dependencies:
  bloc_signals: ^1.0.0
  bloc_signals_flutter: ^1.0.0
  bloc_signals_riverpod: ^1.0.0

Option A: Expose a `BlocSignal` / `CubitSignal` to Existing Riverpod Widgets

Want to write a new feature or state controller with BlocSignal, but keep your existing Riverpod UI layer (ConsumerWidget, WidgetRef)?

Simply call .toProvider():

import 'package:bloc_signals_riverpod/bloc_signals_riverpod.dart';

// Create your new BlocSignal or CubitSignal
final todosCubit = TodosCubit();

// Convert it directly into a Riverpod NotifierProvider!
final todosProvider = todosCubit.toProvider();

// Now consume it anywhere in existing Riverpod widgets:
class LegacyRiverpodWidget extends ConsumerWidget {
  const LegacyRiverpodWidget({super.key});

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    // Rebuilds reactively whenever todosCubit emits!
    final todos = ref.watch(todosProvider);

    return ListView.builder(
      itemCount: todos.length,
      itemBuilder: (context, i) => Text(todos[i].description),
    );
  }
}

Option B: Adapt an Existing Riverpod Provider into `BlocSignal`

Have a legacy Riverpod provider that you need to read from a new BlocSignalBuilder widget?

Use .toBlocSignal(ref):

import 'package:bloc_signals_riverpod/bloc_signals_riverpod.dart';

class NewFeatureWidget extends ConsumerWidget {
  const NewFeatureWidget({super.key});

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    // Adapt any Riverpod ProviderListenable into a BlocSignal container!
    // Automatically binds ref.onDispose to close the container when disposed.
    final todosBloc = legacyRiverpodProvider.toBlocSignal(ref);

    return BlocSignalBuilder<BlocSignalBase<List<Todo>>, List<Todo>>(
      bloc: todosBloc,
      builder: (context, todos) {
        return ListView.builder(
          itemCount: todos.length,
          itemBuilder: (context, i) => Text(todos[i].description),
        );
      },
    );
  }
}

Converting `AsyncValue` <-> `AsyncState`

bloc_signals_riverpod also provides extension methods to map seamlessly between Riverpod's AsyncValue and Signals' AsyncState:

// Riverpod AsyncValue to Signals AsyncState
final asyncState = riverpodAsyncValue.toAsyncState();

// Signals AsyncState to Riverpod AsyncValue
final asyncValue = signalsAsyncState.toAsyncValue();

Explore the Official Riverpod Ports on blocsignal.dev

To prove the DX gains and side-by-side equivalence, we've ported canonical state management examples directly from the official rrousselGit/riverpod monorepo into the BlocSignal open-source repository:

📝 Riverpod Todos (examples/riverpod_todos):
- Replaces @riverpod codegen and Notifier with CubitSignal<List<Todo>> and synchronous computed() signals for reactive filter tabs and stats.
🔍 Pub.dev Package Search (examples/riverpod_pub):
- Replaces Riverpod AsyncNotifier with BlocSignal and a streamless restartable() event transformer that automatically cancels in-flight API requests on keypresses without Rx streams.
🦸 Marvel Character Browser (examples/riverpod_marvel):
- Demonstrates API pagination, character search, and widget tree scoping via BlocSignalProvider.value.

Explore all 20 side-by-side benchmark ports across BLoC, Signals, and Riverpod live at blocsignal.dev/#ported-examples!

Built-In AI Agent Skills for Automated Migration

If you use AI coding assistants like Antigravity, Gemini, Cursor, or GitHub Copilot, BlocSignal publishes a dedicated Agent Plugin skill bundle (riverpod_migration.md).

When your AI assistant inspects a project with BlocSignal skills enabled, it automatically understands:

How to map StateNotifierProvider / NotifierProvider to CubitSignal;
How to preserve auto-disposal and cancellation contracts;
How to refactor ConsumerWidget rebuild boundaries to BlocSignalBuilder or SignalBuilder;
How to apply bloc_signals_riverpod interop adapters during multi-phase refactoring.

Summary

You don't need to throw away your existing architecture to enjoy the speed, simplicity, and zero-codegen elegance of reactive signals.

With bloc_signals_riverpod, you can trial BlocSignal on a single screen today, bridge your existing Riverpod providers seamlessly, and upgrade your developer experience at your own pace.

🌐 Website & Comparison Benchmarks: blocsignal.dev
📦 Pub.dev Packages: bloc_signals | bloc_signals_flutter | bloc_signals_riverpod
🐙 GitHub Repository: RandalSchwartz/BlocSignal

What to Expect From Pure CPU-Only Local Inference

Multigrid — Wed, 12 Aug 2026 22:56:25 +0000

Nobody publishes tokens per second for your model on your RAM, and any page that gives you one has either measured a machine that is not yours or made it up. What can be established is a ceiling, from two numbers you can look up in five minutes.

Two phases with different bottlenecks

A request has a prefill phase, where the whole prompt goes through the model in one parallel pass, and a decode phase, where each output token requires its own pass. They are limited by different things, and conflating them is why CPU inference advice is so often useless.

Decode is the one people mean when they say a local model is slow. To produce one token, the CPU must read every active weight in the model out of RAM and into cache, do a small amount of arithmetic with each, and move on. At a batch size of one there is nothing to amortise that read against — one pass over all the weights buys exactly one token. So decode is bound by memory bandwidth, and the arithmetic units are mostly idle waiting for data.

The bandwidth ceiling, derived

If each token requires reading the weights once, then:

tokens/second  <=  usable memory bandwidth (bytes/s)  /  weight bytes

The numerator is derivable from the DDR standard your machine uses. A DDR4 or DDR5 channel is 64 bits wide, so it moves 8 bytes per transfer, and the number in the name is the transfer rate:

DDR4-3200, one channel:  3200e6 transfers/s x 8 B  = 25.6 GB/s
DDR4-3200, dual channel:                        = 51.2 GB/s
DDR5-5600, one DIMM:     5600e6 transfers/s x 8 B  = 44.8 GB/s
DDR5-5600, two DIMMs:                              = 89.6 GB/s

The denominator is the size of the file on disk, which you can read off the repository listing before downloading. It is a good proxy for the bytes streamed per token in a dense model, because a GGUF file is almost entirely tensor data.

Apple publishes unified memory bandwidth directly rather than leaving you to derive it: in Apple’s October 2024 announcement of the M4 Pro and M4 Max the figures given are 273 GB/s for the M4 Pro and 546 GB/s for the M4 Max. That is the single reason Apple Silicon does well at local inference — not the CPU, the bus.

The ceiling for common configurations

Dividing the bandwidths above by real published file sizes gives the following ceilings. Every file size is from the corresponding Hugging Face GGUF repository listing, checked 2026-08-11; every bandwidth is either derived above or published by Apple. Every cell is bandwidth divided by file size and nothing else.

Model (Q4_K_M)             size      DDR4-3200   DDR5-5600   M4 Pro    M4 Max
                                     51.2 GB/s   89.6 GB/s  273 GB/s  546 GB/s
Phi-3.5-mini-instruct      2.39 GB      21.4        37.4      114.1     228.1
Meta-Llama-3.1-8B-Instr.   4.92 GB      10.4        18.2       55.5     111.0
Qwen2.5-Coder-14B-Instr.   8.99 GB       5.7        10.0       30.4      60.7
Meta-Llama-3.1-70B-Instr. 42.52 GB       1.2         2.1        6.4      12.8

  units: tokens per second, upper bound only

Read the bottom row carefully, because it is the useful one. A 70B at Q4_K_M on a dual-channel DDR5 desktop cannot exceed roughly two tokens per second no matter what CPU is attached, because 42.5 GB has to cross the memory bus for every single token. No amount of cores fixes that. It is also why the 70B row is the one where people conclude local inference does not work, when what does not work is that specific ratio.

These are ceilings, not predictions, and they are the only kind of number that can honestly be given without running your machine. A real figure requires a measurement on your hardware — see benchmarking your own tokens per second.

Why you will not hit the ceiling

Achieved bandwidth is always below peak, and on a general-purpose desktop it is well below. The gap has named causes, all of which you can check on your own machine rather than take on trust:

Peak is a transfer rate, not a delivery rate. Real access patterns pay for row activation, refresh cycles and bank conflicts. A STREAM-style benchmark on your own machine gives the achieved figure, and it is the number to put in the numerator if you want the ceiling to mean anything.
Single-channel configurations halve it. A laptop with one DIMM populated, or soldered memory on a narrow bus, gets half the table above. This is the most common reason a machine underperforms its spec sheet, and it is checkable with dmidecode on Linux or wmic memorychip on Windows.
Thread count stops helping early. Once enough threads are issuing loads to saturate the controller, more threads add contention. This is why llama.cpp’s -t often peaks below the core count, and why the right value is found by sweeping it rather than by setting it to the number of cores.
Weights are not the only traffic. The KV cache is read every token too, and it grows with context. On a long conversation this becomes a meaningful share of the bytes moved — see how the KV cache works for the mechanism.

Prompt processing is a different number

Prefill is not bandwidth-bound, because the whole prompt is available at once and every weight read is reused across all of its tokens. It is bound by arithmetic instead, at roughly two floating-point operations per parameter per token:

8.03e9 params x 2 FLOP  =  16.06 GFLOP per token of prompt
512-token prompt        =  8.2 TFLOP total

Your CPU’s peak throughput is derivable the same way: cores x FMA units x SIMD lanes x 2 x clock. A sixteen-core machine with two AVX-512 FMA units doing sixteen FP32 lanes at 3.5 GHz peaks at about 3.6 TFLOP/s, which puts that 512-token prefill at roughly 2.3 seconds before any inefficiency — several seconds of silence before the first token, on a machine that then generates at a perfectly usable rate.

The practical consequence is that CPU-only inference degrades on prompt length far more sharply than on output length, which is the opposite of the intuition people bring from hosted APIs. Short prompts and a small model are comfortable. Long documents are not, and no quantization level fixes that, because prefill cost scales with parameters and tokens, not with bits per weight.

Writing a Contract Test Suite for Your Own LLM Gateway

Multigrid — Wed, 12 Aug 2026 22:50:22 +0000

If you run a proxy in front of one or more model providers, the interesting failures are not in the model. They are in your layer: a header you forwarded that you should have stripped, an upstream 429 you turned into a 500, a stream you accidentally buffered.

The layer under test

A gateway sits between your applications and one or more providers, and it promises two things at once. Downstream, it promises to speak the OpenAI-compatible contract so that ordinary SDKs work against it. Upstream, it promises to translate faithfully into whatever each provider actually wants. Almost every bug that costs you an incident lives in one of those two promises rather than in the inference.

That means the suite you want is not the one from evaluating a candidate provider. That suite asks whether a remote service is shaped correctly. This one asks whether your code preserves shape, and it should run without touching a provider at all: fast, deterministic, and on every pull request. The distinction matters because a suite that needs a real model is a suite that runs nightly at best, and a proxy bug found nightly has already shipped.

List the promises explicitly before writing a line. A typical set: the response envelope is preserved; the model alias resolves to the documented concrete model; the upstream key never appears downstream and the downstream key never appears upstream; usage is reported and equals what the upstream reported; an upstream error becomes a documented downstream error rather than a stack trace; and a streamed request produces a stream, not a buffered response delivered at the end.

Stub the upstream, not the gateway

The direction of the stub is the design decision. If you mock your own gateway you are testing nothing. What you want is a fake provider: a local HTTP server that returns whatever fixture the test needs, including malformed and hostile fixtures a real provider will not produce on demand. Mock Service Worker’s Node interceptor is the usual choice in TypeScript, and it works at the request level so your gateway’s real HTTP client, real timeouts and real retry logic all execute.

// gateway/contract/upstream.ts
import { setupServer } from "msw/node";
import { http, HttpResponse } from "msw";

export const upstream = setupServer();

const FIXTURE_ID = "chatcmpl-fixture-1";

export const completionFixture = (over = {}) => ({
  id: FIXTURE_ID,
  object: "chat.completion",
  created: 1754870400,
  model: "provider-model-v2",
  choices: [{
    index: 0,
    message: { role: "assistant", content: "ok" },
    finish_reason: "stop",
  }],
  usage: { prompt_tokens: 11, completion_tokens: 2, total_tokens: 13 },
  ...over,
});

export const respondsWith = (status: number, body: unknown) =>
  http.post("https://upstream.test/v1/chat/completions", () =>
    HttpResponse.json(body as never, { status }));

Two properties make this worth the setup. The fixture is a value, so a test can mutate one field and assert that your gateway notices. And the interceptor records the outbound request, so you can assert on what your gateway sent, which is half the contract and the half that is otherwise invisible.

Pass-through fidelity

The single most valuable assertion in a gateway suite is that unknown fields survive. A gateway that parses an upstream response into a typed struct and re-serialises it will silently drop every field its struct does not know about — and providers add fields constantly. Your users lose logprobs, or a reasoning block, or a cache-hit count, and nobody notices until one of them opens a ticket.

it("preserves fields the gateway does not model", async () => {
  upstream.use(respondsWith(200, completionFixture({
    system_fingerprint: "fp_abc123",
    provider_specific_metadata: { cache_hit: true },
  })));

  const res = await fetch(gatewayURL + "/v1/chat/completions", {
    method: "POST",
    headers: { "content-type": "application/json", authorization: "Bearer tenant-key" },
    body: JSON.stringify({ model: "alias-fast", messages: [{ role: "user", content: "hi" }] }),
  });
  const body = await res.json();

  expect(body.system_fingerprint).toBe("fp_abc123");
  expect(body.provider_specific_metadata).toEqual({ cache_hit: true });
  expect(body.usage.total_tokens).toBe(13);
});

The companion assertion runs in the other direction: capture the request your gateway made upstream and check that the client’s parameters arrived intact. A gateway that rebuilds the request body from a known-parameter list drops the one parameter a client added last week, and the client sees a model that ignores their response_format with no error anywhere.

Error mapping and header hygiene

Error mapping is a contract in both directions and it is the part teams write once and never test. Assert each upstream status your gateway can receive against the downstream status it should produce: a 429 must stay a 429 or your clients’ backoff never engages; a Retry-After header must survive, since it carries the only number worth obeying; a 400 from the provider about an invalid parameter must not become a 502, because a 502 tells the client to retry a request that will fail identically forever.

Upstream 429 with Retry-After: 3 → downstream 429 with the same header. Assert the header, not only the status.
Upstream 401 → downstream 502 or 500, never 401. A 401 tells your tenant their key is bad, and it is not — yours is.
Upstream timeout → downstream 504 within your own documented deadline. Assert the elapsed time is bounded, or the test passes while the behaviour is unbounded.
Upstream returns HTML — a CDN error page rather than JSON — → a structured error, not a JSON parse exception. This is a real production shape and worth a fixture.

Header hygiene deserves its own explicit test because the failure is a security failure rather than an availability one. Assert that the downstream Authorization value never appears in the captured upstream request, and that no upstream provider header that identifies your account is echoed downstream. Both are one-line assertions on captured objects and neither is caught by any other kind of test.

Streaming is a separate contract

A gateway can pass every buffered test and still be broken for streaming, because the two are usually different code paths and the streaming one has a failure mode the buffered one cannot have: it can be correct in content and wrong in time. A proxy that reads the whole upstream stream, assembles it, and writes it out as SSE at the end produces byte-identical output and destroys the only reason anyone streams.

So assert on timing, not only on bytes. With a stub upstream you control the delay between chunks, which makes this deterministic rather than flaky: emit chunk one, wait, emit chunk two, and assert that your gateway delivered the first chunk before the second was sent. Record the wall-clock offset of the first downstream chunk and require it to be below the upstream’s inter-chunk delay. The detail of the wire format — the data: prefix, the blank line separator, the terminating sentinel — is covered in contract tests for streaming chunk format, and a gateway suite should reuse the same parser its clients use rather than writing a second one.

Most of the work in operating a gateway is exactly this suite: the error mapping, the header hygiene, the pass-through fidelity, and the streaming path, maintained across every provider you add. If that layer is not itself the product you are building, Multigrid is that layer as a service — one API and one key, with the provider differences absorbed behind it.

I Un-Killed a 1.2k-Star Reddit Downloader That Reddit's API Changes Broke

hideki — Wed, 12 Aug 2026 22:48:59 +0000

tl;dr — RedditDownloader (RMD) was a solid tool for archiving Reddit media. In July 2023, Reddit killed Pushshift and locked down direct API access. The maintainer archived the project and explicitly invited forks. I replaced the dead data layer with the Sylvia API (sylvia-api.com) and got subreddit scanning working again. The fork is at github.com/c1nn3r/RedditDownloader.

The problem

RMD was well built. The repo has about 1,200 stars. It shipped a real filter engine, a local web UI, image-hash duplicate detection, a sqlite state database, and resume support so a large archive run can stop and start without re-downloading.

It scanned many places for media: subreddits, user posts, saved lists, upvoted lists, and multireddits. It extracted links from post URLs, from self-post text, and from comments. It handled most of the hosts you meet on Reddit: i.redd.it, v.redd.it, preview.redd.it, imgur, gfycat, and tumblr, with yt-dlp in the corner for video. It ran headless or as a server with a web UI. For the data-hoarding crowd — the people who believe the internet is ephemeral and the stuff they like will vanish — it was the tool.

Then Reddit changed the rules. In July 2023, Reddit destroyed Pushshift, which RMD used for historical data. Reddit also restricted direct API access and made OAuth app registration slower and stricter. The maintainer, ShadowMoose, wrote a blunt shutdown note. He said that the changes made it impossible for RMD to operate. He said that he no longer wanted to build on Reddit's platform at all. Then he archived the repo. But he left the door open:

"Other users are welcome to fork the project and continue onward, if anybody is willing to continue in spite of the Admin actions pushing against them."

So I forked it.

What was actually broken

RMD splits cleanly into two halves: sources and processing. A source decides where posts come from. Processing decides what happens to a post once you have it — filtering, dedup, download, file naming.

The processing half was healthy. The filters, the downloaders, the image-hash dedup, the sqlite state — none of that depended on the API changes.

The source half was dead. Subreddit and user scanning depended on two things that no longer worked:

Pushshift — offline. The API is gone and the Python wrappers around it (psaw, pmaw) are orphaned.
Direct PRAW / JSON access — still alive, but the cost increased. You need your own OAuth app registration, which Reddit approves slowly and strictly. Rate limits come back as HTTP 429 responses, which make bulk scanning painful. And if Reddit has flagged your IP, the HTML and JSON endpoints just serve you a block page.

Everything downstream of the data layer was fine. The rot was entirely in how RMD got post data in the first place.

The fix

I did not touch the processing pipeline. The source/processing split is the reason the fix is small: I wrote one new source.

The new source is SylviaSubredditSource. It replaces the dead Pushshift source for subreddit listings. It pulls posts from the Sylvia API, a Reddit data gateway that skips OAuth app registration entirely.

The flow looks like this:

Sylvia API (subreddit posts)
    → SylviaSubredditSource
    → RedditElement  (the same format RMD always used)
    → direct_link handler → filters → downloaders → disk

How it works:

RMD calls the source with a subreddit name, a sort order (top, new, hot, controversial), and a time window.
The source calls GET /v1/reddit/r/{subreddit}/{order} on the Sylvia API, with the time filter and a limit.
Sylvia returns Reddit-shaped JSON — the same field names Pushshift used to return: id, title, selftext, url, score, num_comments, over_18, created_utc.
The source wraps each post in a dict subclass named Submission.
RedditElement.detect_type() sees the class name, routes the object through the existing Pushshift parsing path, and produces the exact RedditElement object RMD always used.
Everything downstream runs unchanged: the direct_link handler, the filters, the downloader pool, the file naming, the dedup.

Pagination uses the same after-cursor pattern Reddit itself uses. The source loops until it hits the requested limit or the API stops returning posts. That is the same 1,000-post-per-listing window Reddit serves to everyone.

Setup is one environment variable:

export SYLVIA_API_KEY=syl_...

Then add a "Sylvia Subreddit" source in the WebUI or in the settings file, exactly like any other source in RMD. That is the whole migration. You do not need to register a Reddit app, and you do not need to fill in the OAuth fields that have been sitting empty since 2023.

Verification

I did not want to claim that it works without proof. The fork ships two tests:

test_sylvia.py — pulls real posts from r/aww through the Sylvia API, builds RedditElement objects, checks that media URLs are extracted, and checks that the URL filter still works.
test_download.py — end to end. It runs RMD's real direct_link handler against URLs the Sylvia source produced, and confirms that files land on disk.

The download test tried 5 posts and wrote 4 files. The 5th was a self-post with no direct media link, which the handler correctly rejected. Files ranged from 160 KB to 1.3 MB. Every byte of the download path used RMD's real handler code — no stubs, no test doubles.

I also checked the one thing I was worried about: the media CDN. Reddit blocks flagged IPs on its HTML and JSON endpoints, and this machine is flagged. The good news is that the media CDN (i.redd.it, preview.redd.it) does not use the same block list. Downloads worked from this machine. The data layer never touches the blocked endpoints at all, because Sylvia is the one making those requests.

What still does not work

I want to be clear about the limits, because some RMD features are not fixed and cannot be fixed by swapping a data source.

The OAuth-only sources are still broken: saved posts, upvoted lists, multireddits, and "your own user's posts." These need an authenticated Reddit session. That is a property of your Reddit account, not a property of data access. No data gateway can stand in for it. Those sources still need PRAW auth exactly as before.

There is also no deep history. Reddit's JSON API serves only a window of about 1,000 posts per listing, and Sylvia proxies that same window. The Pushshift-era ability to pull "every post ever made in this subreddit" is gone for everyone, not just for this fork.

Media binaries still download from Reddit's CDN. That worked in my tests. If Reddit later blocks download IPs the way it blocks the HTML and JSON endpoints, you will need a proxy for the media fetch — but the data layer will keep working.

If you used RMD for subreddit archiving, this fork gets you back to working. If you relied on saved or upvoted-list syncing, this does not help you yet. That is a different problem — authentication, not data access — and I have not tackled it.

Why fork instead of pull request

The original repo is archived. The maintainer said, plainly, that he is done with anything Reddit-related. A pull request would sit in the queue forever. He explicitly invited forks in the shutdown note, so that is the route I took.

Credit goes where credit is due. The architecture, the handlers, the filters, the dedup, the web UI — that is ShadowMoose and the contributors' work, and it is good work. I replaced one dead pipe. That is the whole diff, and it stayed that small because RMD was built well enough to separate "where the data comes from" from "what the data does."

Made possible by Sylvia API

Sylvia API is what made this fork possible. When Reddit killed Pushshift and locked down PRAW access, RedditDownloader's subreddit-scraping path died with them. Sylvia is a Reddit data gateway that serves Reddit JSON with no OAuth app registration, no rate-limit roulette, no 429s, no IP blocks — just a key and clean JSON. One source swap restored RMD's subreddit scan, filter engine, and media download pipeline for archiving, data hoarding, and image/video backups.

Credit where credit is due: the architecture is ShadowMoose's. Sylvia is the pipe that made it run again.

Fork: github.com/c1nn3r/RedditDownloader