# Scoring models (/concepts/scoring-models)



A scoring model rates the risk of one kind of decision, such as a payment, a sign-up or a loan application. You create it once, from a description of the decision or from past records whose outcomes you know, score each new record with [`POST /v1/score`](/reference/score), and calibrate it with outcomes as you learn them so it keeps improving. Each scoring model is your own. It scores a record by asking Grayson a handful of yes/no questions about it and weighing the answers ([Questions and weights](#questions-and-weights)).

The score is an integer from 0 (lowest risk) to 1000 (highest). Once the scoring model is calibrated, divided by 1,000 it's about the probability that the outcome is bad: of records scored near 250, about 1 in 4 turn out bad. Before then, scores rank records, but their scale is an estimate. See [Calibration](/concepts/calibration) for what that rests on and how to choose thresholds.

## Creating a scoring model [#creating-a-scoring-model]

Create one with [`POST /v1/scoring_models`](/reference/create-scoring-model), from either:

| From             | Send                                                                                                     | You get                                                                                                                               |
| ---------------- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Its criteria     | `name` and `criteria`: what the decision is and what a bad outcome means                                 | Questions Finic writes from the criteria, with weights it estimates. Ready in less than a minute, and it scores any record right away |
| Labeled examples | `name` and `data`: past records with `is_bad` ([Example data](#example-data)), and optionally `criteria` | A scoring model calibrated to your examples from the start. Ready in about two minutes                                                |

```bash
curl https://api.finic.ai/v1/scoring_models \
  -H "Authorization: Bearer $FINIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "Card-not-present payment fraud", "criteria": "Card-not-present payments at an online store. Bad: the payment was charged back as fraud within 90 days."}'
```

A scoring model created from its criteria isn't calibrated yet:

* **Its weights are Finic's estimates,** and `calibrated_at` is `null`. Its scores still run from 0 to 1000 and rank records, but their scale is an estimate, not calibrated to your rate of bad outcomes: choose thresholds once it's calibrated.
* **It takes any record,** since it has no [schema](#schema) until you calibrate it with examples.
* **You can add and remove its questions** freely ([Changing questions](#changing-questions)).
* **Calibrate it** with labeled examples, or with the outcomes of at least 100 records it scored, 20 of each ([Calibration](/concepts/calibration#a-scoring-models-first-calibration)).

## Example data [#example-data]

To create a scoring model from examples, or calibrate one with them, send past records, each with `is_bad`: `true` when the outcome was bad, `false` when it was good. Send them as a JSON array of objects (nested objects and arrays are fine) or as CSV with a header row, as text or a file:

```csv title="examples.csv"
amount_usd,merchant_category,account_age_days,new_device,billing_country,ip_country,is_bad
2400.00,electronics,19,true,US,NG,true
58.12,grocery,1260,false,US,US,false
310.00,gift_cards,3,true,US,US,true
```

### Rules [#rules]

* **At least 100 rows, with at least 20 of each outcome.** Up to 2,000 are used; a larger set is sampled down, keeping its mix of outcomes. The request can be up to 2 MB.
* **`is_bad`** is `true` or `false`, `1` or `0`, or the text `"true"`, `"false"`, `"1"`, `"0"`, `"yes"` or `"no"`, in any case.
* **Every row has at least one other field,** and is up to 60,000 bytes as compact JSON without `is_bad`.
* **CSV:** every row has as many columns as the header, which names each column once. An empty cell is `null`, `true` and `false` are booleans, plain numbers are numbers, and the rest is text (leading zeros, as in ZIP code `02139`, stay text). A dotted column name is a nested field: `merchant.category` is `{"merchant": {"category": …}}`, so CSV examples match JSON records scored later. A dotted name that clashes with another column stays one field.
* **`name`** (required, up to 100 characters): a few words for people to recognize the scoring model by, such as "Card-not-present payment fraud". You can rename it in the Finic Portal.
* **`criteria`** (up to 4,000 characters): what the decision is and what `is_bad` means, such as "Card-not-present payments. is\_bad: the payment was charged back as fraud within 90 days." Required without `data`, since Finic writes the questions from it; with `data`, optional, and used as context.
* **`add_questions`** (optional, up to 20, 500 characters each): your own yes/no questions to use alongside Finic's. See [Changing questions](#changing-questions).

### What to include [#what-to-include]

* **Customer identity or event data in JSON:** sign-ups, logins, payments, transactions, applications. One row per past decision, with its real outcome as `is_bad`.
* **The fields you'll send when you score,** with the same names and shapes, from the same systems.
* **Only what was known at the time of the decision.** A later chargeback, fraud flag or analyst verdict makes the scoring model look better than it will be on new records.
* **Readable, consistent fields:** names in words (`amount_usd`, not `amt`), units and currencies, codes expanded, the same type for a field in every row.
* **Recent, representative records** from the population you'll score.

## Schema [#schema]

Creating a scoring model from examples, or calibrating one with examples for the first time, records its schema: every field in the examples and its type (text, number, boolean, object or array). Nested fields have paths such as `account.age_days`, and fields in arrays paths such as `logins[].country`. A field every example has, never `null`, is required.

Each record you score is checked against it and rejected with `400 schema_mismatch` if a required field is missing or `null`, a field is new, or a field has a type the examples never had. Text counts as the same type as numbers and booleans, so `"58.12"` fits a number field. The error lists up to 20 differences:

```json title="400"
{ "error": { "type": "schema_mismatch", "message": "data doesn't match the scoring model's schema: missing field: account_age_days; unexpected field: account_age. Send skip_schema_validation to score it anyway, or update_schema to make this the new schema.", "param": "data" } }
```

| Flag on `POST /v1/score` | Does                                                                                                                                                 |
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `skip_schema_validation` | Scores this record without the check; the schema doesn't change. For one-off records you know differ                                                 |
| `update_schema`          | Scores this record without the check and makes its fields the schema from then on. For a lasting change, such as a new field; send a complete record |

A scoring model created from its criteria has no schema, and takes any record, until you calibrate it with examples (or a score call sends `update_schema`).

A scoring model is most reliable on records like its examples. If your data changes a lot, create a new one.

## Score ids [#score-ids]

Every score call carries a `score_id`: your own id for that record, such as the payment's id, up to 200 characters. It's unique within the scoring model: sending it again gets `409 duplicate_score_id`. Keep it with the record, and send it back with the outcome to [calibrate](/concepts/calibration#calibrate-a-scoring-model) the scoring model.

## Questions and weights [#questions-and-weights]

Finic writes questions and uses the ones that best predict your outcomes, and you can add your own. [`GET /v1/scoring_models/{model_id}`](/reference/get-scoring-model) lists the questions a scoring model uses:

```json title="200 (questions only)"
"questions": [
  { "id": "q004", "text": "This record is a card payment. Is the IP address in a different country from the billing address?", "source": "finic", "weight": 0.91 },
  { "id": "q022", "text": "This record is a card payment. Is this a typical amount for the merchant category?", "source": "finic", "weight": -0.52 },
  { "id": "q051", "text": "Was the account opened less than a week before the payment?", "source": "user", "weight": 0.38 }
]
```

* **The strongest is listed first.** Only the questions it uses are listed; its page in the Finic Portal also lists your own questions it doesn't use, under **Not used**.
* **`weight`** is how much a question moves the score, as of the latest calibration (before one, Finic's estimate): positive when a likely yes raises the score, negative when it lowers it. The larger it is either way, the more the question counts.
* **`source`** is `finic` for a question Finic wrote, `user` for one you added.

To see what's behind one record's score, send `include_questions: true` to [`POST /v1/score`](/reference/score). The response then lists each question the score used, with Grayson's `probability` of yes for that record and the question's `weight`.

## Changing questions [#changing-questions]

Write each question of your own as a yes/no question about one record. Add and remove questions at any time with [`add_questions`](/reference/add-questions) and [`remove_question`](/reference/remove-question), or as you create or calibrate the scoring model:

* **Before it's calibrated,** a question you add counts right away, with a weight Finic estimates. You can remove any question, as long as one is left.
* **Once it's calibrated,** a question you add is asked on every score call and tried at the next calibration, which uses it if it helps: with examples, or with outcomes once 30 of the records it was asked about have one. Until then, and if the calibration doesn't use it, it's listed as not used on the scoring model's page in the Finic Portal, and tried again at later calibrations.
* **Removing a question the calibrated score uses** happens in a calibration, with `remove_questions` and its `id`, so the score is refit without it ([`remove_question`](/reference/remove-question) gets `409 question_in_use`). Your questions it doesn't use can be removed at any time. A removed question isn't asked or tried again.
* **In a calibration with examples,** `add_questions` adds questions too, and the scoring model uses the ones that help. Adding in a calibration needs examples: Finic doesn't keep the records it has seen, so it answers new questions on new labeled records.
* **Up to 50 changes per scoring model a day,** counting calibrations and creating it.

You can have up to 50 questions of your own per scoring model, used or not, adding up to 20 per request; each is up to 500 characters. A question that's blank, too long or already one of the scoring model's, or an id in `remove_questions` it doesn't have, gets `400 invalid_request`.

## Jobs [#jobs]

Creating and calibrating return `202` with a `job_id` at once; the work runs in a job.

* **Creating takes less than a minute** from its criteria, and about two minutes from examples (longer with more). Until then, score calls get `409 scoring_model_not_ready`.
* **Check the job** with [`GET /v1/jobs/{job_id}`](/reference/get-job-status) every 10 seconds or so, until `complete` is `true`.
* **A failed job** has an `error` saying what to do. A failed creation leaves the scoring model unusable: create it again. A failed calibration leaves it as it was.
* **`job_interrupted`** means maintenance interrupted creating the scoring model or calibrating it with examples, which are held only in memory: create it again, or send the examples again.
* **During a calibration,** the scoring model keeps scoring with its current version.

New scoring models are limited to 100 per organization per day, and calibrations to 10 per scoring model per day; see [Limits](/reference#scoring-models).

## In the Finic Portal [#in-the-finic-portal]

The [Scoring models](https://portal.finic.ai/scoring-models) page lists your scoring models. There you can:

* **Create** a scoring model from its name and criteria.
* **See its questions** and their weights on its page, with a **Not used yet** section for your own questions it doesn't use, and **add or remove questions** there.
* **Calibrate** a scoring model from a CSV or JSON file of labeled examples or of outcomes, removing questions the score uses if you like.
* **See how each calibration did** as a precision-recall curve ([how it's measured](/concepts/calibration#how-each-calibration-did)).
* **Archive** a scoring model to stop calls to it without deleting anything: [`score`](/reference/score) and [`calibrate`](/reference/calibrate) return `409 scoring_model_archived` until you unarchive it.
* **Delete** a scoring model (admins only). Its records and outcomes are deleted, and calls to it return `404 scoring_model_not_found`. This can't be undone.

## What's stored [#whats-stored]

A scoring model keeps its name, its criteria, its schema (field names and types) and its questions, including the ones you added. Each score call keeps its `score_id`, its score, Grayson's answers to the questions and, once you calibrate, the outcome. Each example, whether sent to create the scoring model or to calibrate it, is kept the same way: its answers and its label. The examples and records themselves are processed in memory and aren't stored, whatever your [data retention](/concepts/data#data-retention). Your scoring models are listed by [`list_scoring_models`](/reference/list-scoring-models), and in the Finic Portal under [Scoring models](https://portal.finic.ai/scoring-models).