Case study

From Class Pathways to Degree Pathways

A college-major explorer I built while helping my own kids think about academic paths, the careers they open, and what graduates of a major actually earn. Six forms, one pivot that killed the feature the product was named after, a validator that kept refusing to ship, and user testing that changed both the search box and my mind about who this is for. This is the whole build.

6 forms tried
2.5 months
142 schools live
8,900 field pages
4 federal datasets
$9.99 one-time pass
0 fabricated figures
Solo build

An earnings chart for Computer Science at the University of Arizona showing what graduates earned one and four years out beside the national median for the field today, with source links to College Scorecard and the Census — What the whole product is for: one major, at one school, with every figure carrying its date and a link back to the federal file it came from.

Why I built it

Choosing a major is one of the most expensive decisions a family makes, and most families make it on hearsay. A cousin's opinion, a ranking that measures selectivity rather than outcomes, a brochure. Meanwhile the U.S. government already publishes what graduates of each major at each specific school went on to earn, and which occupations each field of study prepares people for. It sits in IPEDS, in the College Scorecard, in O*NET, in Census tables, in formats nobody outside a research office is going to open.

I wanted my own kids to be able to see it. Not a ranking and not advice, just the record: here is what people who took this exact major at this exact school were earning four years later, and here is the government's own list of the careers it opens.

Six forms before the product existed

The first few weeks were not building, they were elimination. Each version was killed by a real constraint rather than a change of taste, and the sequence is the part worth keeping.

Form 1A downloadable prompt pack. Too much copy and paste friction. A parent will not do it.
Form 2A Claude Project. Too much setup before any value appears.
Form 3A self-installing skill. Requires a paid plan, plus the trust friction of downloading something from a stranger.
Form 4A pure cited course sequencer. I dropped the career rubric to make it narrower and more defensible. Stronger, and the first version I believed in.
Form 5An interactive radial branch wheel. More game than tool, then rebuilt to be genuinely non-linear.
Form 6A desktop Chrome extension wrapping the student's own live catalog view, then on-device AI for universal parsing. This one hit a wall that mattered.

On-device AI would have given me universal catalog parsing at no cost to me and none to the user. It also assumed the student owns a machine that can run a local model. Most do not. That was not a bug to work around, it was the shape of the problem finally becoming visible.

The constraint that forced everything

Three requirements were fixed from the start. It had to serve a non-technical audience, students and the parents helping them. It could never scrape, store or redistribute a school's copyrighted catalog. And it had to run with no API keys, no per-use cost, and no hardware requirement.

The constraint triangle. You can have an AI that reads any catalog, or free to run with no key, or no hardware floor for the user. You cannot have all three at once.

Naming that triangle was the real output of the first month. Every architecture decision after it is just a choice about which corner to give up.

I gave up the first corner. If no AI reads a catalog at runtime, the data has to be assembled in advance, which means it has to come from a source that is complete, uniform, public domain and safe to redistribute. That requirement is what eventually pointed at federal data, though it took a failure to get there.

Locking the architecture

The decision that followed was to build a hosted web app rather than a Custom GPT. A GPT requires the user to have a ChatGPT account, which is a real barrier for a parent, and it cannot hold a license key or meter usage, so it cannot be sold. The cost worry that had pushed me toward a GPT turned out to be solvable another way: the prototype had already proved the matching logic needed no live AI at all. It was a tag-scoring engine over structured data.

So the runtime became fully deterministic and the AI moved offline into a batch pipeline that fills a facts database. No per-user tokens, no model in the live path, nothing to drift. Copyright was handled by storing facts only, meaning codes, titles, credits and figures with source links, never prose descriptions. Facts are not copyrightable. Prose is.

The pivot: when QA killed the feature the product was named after

The whole product was a term-by-term course planner. I ran the generated plans through adversarial QA against real published catalogs, and they came back wrong in ways no amount of prompt tuning was going to fix.

Credits did not reconcile. One mechanical engineering plan claimed 133 credits and displayed 48.
Required courses were missing. CE 214 Statics was absent from a plan that listed it as a prerequisite for a later course.
Sequences inverted. Nursing Management II scheduled before Nursing Management I. Calculus I, II and III in a single term.
Prerequisites at one school were stored as internal IDs that could not be resolved to anything verifiable.

The root cause was structural rather than fixable. Catalog requirement data cannot reconstruct the four-year sequence a school's advisors actually recommend. That sequence lives in advising, and often in a PDF that is not machine readable.

If it is not accurate in its entirety, it does not ship.

Nothing was deployed. I cut the feature the product was named after, and Class Pathways became Degree Pathways.

The useful part was underneath the failure. Course-level data is incomplete and school-specific. Program-level federal data is complete, uniform across roughly 6,000 institutions, public domain and safe to redistribute. The validated pieces of the old product survived the pivot: the interest quiz, the O*NET career matching, the crosswalk from occupations to fields of study. What changed was the layer they attached to.

Rebuilding on federal data, and not trusting the codebooks

Four federal sources, joined offline. IPEDS for which majors each school actually awards degrees in and how many students graduate. The College Scorecard for what those graduates earned, built from IRS records. O*NET for the occupations each field prepares people for. Census for what the field pays nationally today.

The interesting work was not the joining, it was refusing to trust anything I had not verified against live data. Three examples of the same discipline.

The award code that did not exist

I wrote the IPEDS adapter against the documented NCES scheme, where award level 5 means a bachelor's degree. Running it against Arizona returned nothing, because the portal I was pulling from uses a completely different code set with no code 5 in it. The adapter's self-check refused to write rather than producing an empty file, which is the guardrail doing its job. I then identified the bachelor's code empirically, by ranking award levels by completion count at a four-year institution and checking that the resulting top majors looked like real top majors. Award level 7, confirmed independently at Arizona and at Minnesota.

The leading zero

CIP codes are stored as integers in that API, so 01.0999 arrives as 10999 with the leading zero silently dropped. Every code in the agriculture family would have been misfiled into a different discipline. The fix is one line of padding. Finding it required printing real rows rather than reading documentation.

The cross-check that could have shipped wrong counts

Completion counts are broken out by race and sex, and the server ignores the filter that is supposed to return grand totals. So the extractor pulls the total cells and independently sums the disaggregated cells, then hard stops if the two disagree by more than two percent. Graduate counts are one of the few numbers a school will notice being wrong.

The same probe-first rule applied everywhere: never finalize a field mapping from a codebook, always from a real run. Thirteen offline fixture tests for the IPEDS adapter, nineteen for the earnings adapter, twenty-four for the merge, all green before the first real extraction.

Two gates before anything ships

External data is untrusted content. A booby-trapped page could carry prompt injection, scripts, or exfiltration payloads that would land in the database, get served to a student, and be re-read by the pipeline on a later pass. So there are two gates, and nothing reaches the app without passing both.

Quarantine is a security scanner that walks every string in an ingestion file and blocks on prompt injection, role reassignment, jailbreak patterns, scripts and event handlers, SQL injection, dangerous link schemes, non-HTTPS sources, hidden bidirectional and zero-width characters, and leaked secrets. I validated it in both directions: clean on 229 real strings, and twelve catches on a deliberately malicious file I wrote to attack it.

Validate is the copyright and sourcing gate. Facts only, no prose fields, and a working source link on every record. The first time I ran it against the real Arizona seed data it failed all 46 records, because per-course catalog links had not been attached yet. That failure was the gate proving it worked, and it surfaced the actual publish blocker instead of letting it through.

By launch this had grown into a three-stage integrity gate running on every commit: a static scan for any code that could fabricate a fact, a render check that traces every on-screen dollar figure back to its source file, and a recompute of every derived value. It blocks Math.random outright, which is why the "Surprise me" button uses the clock. It currently checks 8,765 figures across 142 schools on each run.

What it shows

Pick your school and you get every undergraduate field it actually awards degrees in. For each one: what its graduates earned one, four and five years out; what that field pays nationally today, shown beside the school's record and never blended into it; the careers the Department of Labor's crosswalk links to that field; median graduate debt; and how many degrees the school awarded last year.

When a program has too few graduates, the government withholds its school-specific earnings figure to protect privacy. In that case the app shows the national median for the same major, clearly labeled as a benchmark. It never fabricates a school figure to fill the hole.

The Degree Pathways browse grid grouped by subject area, with agriculture and environment fields listed first, each card showing its median pay — Browsing a school's full catalog, grouped by subject area. Every card carries its pay figure, so scanning does not require opening each one.

Testing rewrote the search box

Browsing was solved early, and I assumed searching was a smaller version of the same problem. Watching people use it proved otherwise.

Students did not search the way the data is organized. They typed broad professions. Dentist. Doctor. Lawyer. Not "Dental Support Services," not "Registered Nurses, 29-1141.00," and not the name of any field of study. They were starting from the job they could picture, and asking the product to work backwards to the degree.

That is a reasonable way to think and the app had no answer for it. So I brought in the complete list of O*NET occupation names, 22,437 official titles, and tied it to the product, so a student can search from either direction: the field they are considering, or the profession they are imagining.

Which immediately created the harder problem. Two requirements pulled against each other. The box had to understand every word a student would actually type, and it had to never connect a career to a major that the government does not connect. Loosen the matching and you start inventing pathways. Tighten it and half the vocabulary hits a wall.

The resolution was to be strict about who is allowed to make each link. Turning a student's word into an occupation or a field of study is a naming problem, and the government has already published its own answer: O*NET's alternate occupation titles, and the official CIP program titles for every field. Turning an occupation into a degree is the crosswalk's job, and only the crosswalk's. So the app bakes those two published lists into lookup maps, uses them purely to interpret the word, then hands off to the federal crosswalk for the actual connection. A word nobody publishes resolves to nothing, and nothing is exactly what the app shows.

The payoff is that a student can now see the shape of a whole path rather than one link in it. Type dentist and the app does not pretend a dentistry major exists for undergraduates. It explains that dentistry is a graduate and professional path, links the Department of Labor record for the occupation, and then shows the fields at her school that she could actually do well in first. Undergraduate field, then graduate program, then profession, with the app honest about which step it can and cannot speak to.

What testing changed

Almost every meaningful improvement in this product came from watching it fail rather than from planning. A partial list, in order.

A design critique softened the product's manners

A design review aimed at students and parents caught things I had stopped seeing. The percentage fit score became word bands, because handing a seventeen-year-old a number that says her interests are a 62 percent match for a career is a judgment the data cannot support. The destructive delete control was a red X on the left, which read as alarming; it became a low-emphasis gray control on the right, with an undo. Long result panels collapsed into accordions. Low-contrast micro-text got fixed. The review also caught a price contradiction I had introduced during a recolor, where the app advertised one price and the checkout charged another.

A chart that told a true story badly

The earnings view originally drew one, four and five years out as a rising series. Those are three separate graduating classes measured in three different years, not one class followed over time, so the five-year figure sometimes sat below the four-year one and read like a pay cut. Every number was correct and the chart still lied. I dropped the five-year bar and made the labels say plainly that these are different cohorts.

An AI score I removed on reflection

An early version ranked majors by how resistant they were to AI displacement. It tested well and I cut it anyway. A risk ranking can steer a student away from something they love, and the honest position is that nobody knows what the next decade does to any occupation. It became a neutral note that ranks nothing, and the pipeline is instructed never to rank by AI risk.

Mobile testing rewrote the comparison view

The side-by-side comparison table worked on a laptop and forced horizontal scrolling on a phone, which is where this actually gets read. It became vertical metric cards, one metric per card, with the bars and the winner visible without scrolling sideways.

Then I opened it the way my daughter would

The last round was the most useful, because I stopped testing features and started using the app as a seventeen-year-old with a question. Three things were still wrong.

The layout asked her to hold two things at once. A list on the left, a panel on the right, and nothing to look at until she clicked.

The earlier two-column layout with a narrow scrolling list of major names on the left and a detail panel on the right — A list on the left, a panel on the right, four filter dropdowns across the top, and not a single number visible until you clicked something.

The rebuilt single-column grid of field cards, each showing the field name and its median pay — One column, one decision at a time. The pay figure moved onto every card, and the filters moved behind a toggle.

The most common search in the product returned nothing. Medicine, law and veterinary work are graduate paths, so no undergraduate major is federally linked to them. True, and completely useless to a student who just typed "doctor" and got a blank screen. The honest answer was not silence, it was an explanation.

Searching doctor in the earlier version returns the message nothing matches those filters, loosen one and try again — "Nothing matches those filters." Technically correct, and it reads like the app is broken.

Searching doctor now returns a note explaining that medicine is a graduate path with no single feeder major, a link to the Department of Labor record, and the school's fields below — It names the path, links the Department of Labor record, declines to invent a feeder major, then hands back the school's real fields to choose from.

A real field the school does not offer looked like a bug. Search "computer engineering" at a school that does not award it and the old build showed the same empty state. But there is a genuine fact available: the field exists in the federal catalog, it has a code, and this school reported no degrees under it.

Searching computer engineering in the earlier version shows the same empty nothing matches message — The same dead end, with no way to tell a typo from a field the school genuinely does not offer.

Searching computer engineering now names the CIP codes, states that no degrees were reported under them at this school, and lists the engineering fields the school does award — The field, its CIP codes, the plain statement that no degrees were reported under it, and the engineering fields this school does award.

Across a corpus of student vocabulary run against the reference school, dead ends fell from 73 to 25, and every one of the 25 survivors is a term the federal data genuinely has no answer for. That number matters more to me than any of the additions. It is the count of times the product has to say "I don't know," and I would rather it be honest and small than zero and invented.

The moment it could have become a ranking

Comparison was the feature people asked for, and it was the one that could quietly undo the whole product. The thing this app exists to replace is the ranking: a number that sorts schools into better and worse and hides everything that actually produced it. The moment I put three schools in a row with a dollar figure under each, I had built a ranking, whether I meant to or not.

Killing the feature was not the answer, because the question is real. A family genuinely wants to know what the same major leads to at the two schools they are choosing between. What the numbers could not be allowed to do was answer a question they had not been asked.

So every comparison screen carries a panel explaining what a gap is actually made of. Not a disclaimer under the fold, and not a hedge that undercuts the figures. It sits directly under the table, it affirms the numbers are real and published, and then it says plainly what they do and do not account for.

The part that took the longest was realising there are two different questions wearing the same clothes. Two majors at one school and one major at two schools differ for almost entirely different reasons, so a single generic caveat would have been useless for both.

The in-app panel headed how to read a gap between majors, explaining that pay follows the occupations a field feeds, that fields leading to graduate school look lower four years out, that hours and full-time rates vary, and that a small program's median rests on fewer people — Between two majors at one school. The one students misread most is the second: a field that commonly leads to graduate school shows a low four-year figure because many of those graduates are still in training, not because the field pays badly.

The in-app panel headed how to read a gap between schools, explaining where graduates work, who enrolled, which employers each campus feeds, and what sits inside one federal field code — Between two schools. The load-bearing line is that the figure measures an outcome, not the school's contribution to it. The College Scorecard has never been a value-added measure, and saying so is what keeps this from being a league table.

Neither panel claims how much any single factor contributes, because no published figure supports that and inventing one would break the rule the whole product is built on. They name the causes and stop. The majors panel closes on the line I would want a seventeen-year-old to actually take away: a major you'll do well in beats a higher median you won't finish.

There is a version of this product that shows the numbers and says nothing, and it would look more confident. It would also be the thing I built this to replace.

"Why can't I just ask an AI to do this?"

Adult students asked this almost every time, and it is the right question. If you are already comfortable prompting a model, a college-major explorer can look like a wrapper around something you can get for free.

The honest answer is that it is not a competitor to AI search, and pretending otherwise would fail the first time someone tested the claim. The wedge was never that AI gets things wrong, because a cited answer from a model can be perfectly right. The wedge is that two people asking the same model the same question get different answers, in different formats, with different sources, and no way to line them up. This product does one narrow thing in a consistent, sourced, repeatable format across every field a school actually awards, with no prompt crafting and no variance.

Because it came up so consistently, I stopped treating it as an objection to handle in conversation and built the answer directly into the marketing page: a side-by-side of what each is genuinely better at. AI explores the questions you bring and goes deep on one idea. This surfaces options you did not know to ask about, keeps the measures and timeframes identical across everything you compare, and puts a source link on every number. The recommended path is both. Build the shortlist here, then take it to an AI, to the school's catalog, and to an advisor for the deeper conversation.

Who it turned out to be for

The most useful thing testing told me had nothing to do with the interface.

I built this for the enrolled 18-year-old choosing a major. She is real and she is the eventual user, but she is not who is thinking about this in August. A student arriving on campus is thinking about decorating a dorm, meeting people, and rush. Asking her to sit down with federal earnings data that week is asking for attention she does not have and should not have to give. She settles into this product later, once she is settled into school and an actual decision is in front of her.

What she does do, immediately and heavily, is lean on her advisor. That reframed the whole thing for me.

College advisors, high school advisors and parents are the conduit to the student. They are the ones holding the question early, they are the ones she brings it back to, and they are the ones who need a sourced, consistent answer they can point at rather than an opinion.

Which makes them a primary audience rather than a secondary one, and it changes what the product has to be good at: not persuading a student, but being something an advisor is willing to put in front of one.

That also explains a few earlier decisions in hindsight. The insistence that every figure carry its source and its date is not a purity exercise; it is what makes the product usable by someone whose professional credibility is on the line when they recommend it. The refusal to rank, to steer, or to sell student contact information is the same. An advisor cannot hand a student a tool that is quietly monetizing her.

Users also asked whether it covers graduate school. It does not yet, and it is the clearest candidate for the next version, especially now that the search understands professions that require a graduate degree and has to stop at the edge of what it knows.

The pricing went four rounds

It started as a $19 one-time pack, moved to $12.99 a year when the product became a hosted app, dropped to $4.99 to cut friction, and landed at $9.99 once the payment processor's flat fee made the math clear. A merchant of record takes five percent plus fifty cents, which is a rounding error on a large purchase and a meaningful bite out of a small one.

What did not change is the shape. Exploring is free and always will be, because a student comparing majors should not hit a paywall. The pass is one payment, good for twelve months on up to five devices, not a subscription and no auto-renew.

I considered and rejected two obvious revenue models. Display advertising needs 50,000 to 100,000 sessions a month before the numbers work at all. College lead generation would pay considerably better, and it is disqualifying: the honest-data positioning cannot coexist with selling student contact information to the institutions whose outcomes the app is reporting.

What runs before anything ships

By launch there were four checks, and all four have caught something real:

The integrity gate recomputes 8,765 derived figures against their sources across all 142 schools and blocks any dollar figure that cannot be traced.
A render sweep loads all 8,900 field pages across all 142 schools and fails on any exception, blank card, or stray NaN. Presentation bugs at scale are invisible to spot-checking.
A coverage guard runs a corpus of student search vocabulary and reports every term as answered or dead-ended, so a regression in search quality shows up as a number rather than as a complaint.
152 search assertions cover the cases that were hardest to get right, including every graduate path that must refuse to name a feeder major.

Using it as a person still caught the kind of bug no automated check was going to. Opening a shared link, rather than picking a school inside the app, skipped the call that loads the two search lookup maps. Search silently degraded to plain title matching for that entire session, which is precisely how a good query turns into a dead end. It had been shipping that way for weeks, and every gate was green the whole time.

On a phone, at the kitchen table

This gets read on a phone, usually with a parent looking over a shoulder. The rebuilt layout is one column with real tap targets, the pay figure on every card so scanning does not require opening each one, and the same source links a laptop gets. No account, nothing installed, and nothing about the student's exploring is stored or sent anywhere.

The earlier mobile layout, where a screen and a half of header and trust badges came before the first major — A screen and a half of header before the first field.

The current mobile layout, with a field card and its earnings visible on the first screen — A real field card, with its number, on the first screen.

Three things I would carry into the next build

Name the who-pays constraint on day one

Who pays for the compute, and on whose hardware, decides your entire architecture. I found that triangle by running into it six times. Naming it first would have saved most of a month.

Give accuracy veto power, and let it cost you a feature

"If it is not accurate in its entirety, it does not ship" sounds expensive, and it is the cheapest call available. It killed the feature this product was named after, and it killed an earnings chart I liked, and both times what replaced it was better. It eliminates the entire class of bug where a wrong number reaches somebody making a real decision about their life.

Build the guardrail before you need it

Every gate in this project caught something I did not expect, usually within days of being written. The award-code check refused to write an empty file. The copyright gate failed 46 out of 46 records. The render sweep found problems on schools I had never opened. None of those were the bugs I built the gates to catch.

Where it goes next

Graduate programs are the most requested addition and the most natural one, because the search already recognizes the professions that require a graduate degree and currently has to stop at the boundary of what it can source. Extending it there closes the loop the product already describes.

Alongside that: more schools as the federal pipeline publishes them, the newest IPEDS and Scorecard releases flowing in automatically, and a version of this built for the advisors and parents who turn out to be the ones carrying it to the student.

Beyond that I am deliberately slow to add. Every feature I have cut from this product was cut for the same reason, which is that it could not be made accurate, and that constraint has been a better editor than any roadmap I could have written.