AI can turn unstructured notes into structured JSON, but the result still needs validation. This guide uses a tiny synthetic example and a Python schema check to verify required fields, types, allowed values, and basic consistency.
Who this is forPeople who use an AI chat assistant to convert notes, emails, forms, or other messy text into structured JSON and need a repeatable validation step.
AI is useful for turning irregular text into structured fields, but a response that looks like valid JSON can still be wrong. A required key may be missing, a number may be returned as text, a field may contain an unsupported value, or the AI may infer information that was never present in the source.
The safest workflow separates extraction from validation. First ask the AI to map the source text into a fixed structure. Then check the returned JSON against explicit rules. Validation does not prove that every extracted fact is correct, but it catches many structural errors before the data is copied into a spreadsheet, database, script, or workflow.
This example is synthetic. Suppose a short service note contains a customer name, ticket number, priority, affected products, and an optional callback time.
Source text: "Ticket 1842. Customer: Mira Lee. Login fails on the desktop app and web portal. Priority is high. Please call after 15:30. Products affected: Desktop, Web."
| Field | Rule |
|---|---|
| ticket_id | Required integer |
| customer | Required non-empty string |
| priority | Required: low, medium, or high |
| products | Required non-empty list of strings |
| callback_time | String in HH:MM form or null |
The schema is deliberately simple and can be checked with the Python standard library. The rules are stricter than merely asking whether the response can be parsed as JSON.
Tell the AI exactly which fields are allowed and how to handle missing information. For example: "Extract the following text into one JSON object with exactly these keys: ticket_id, customer, priority, products, callback_time. ticket_id must be an integer. priority must be low, medium, or high. products must be a list of strings. Use null for callback_time if it is not stated. Do not infer missing facts. Return JSON only."
A correct result for the synthetic note would be: ticket_id 1842, customer Mira Lee, priority high, products Desktop and Web, and callback_time 15:30.
A plausible but defective AI result might instead return ticket_id as the string "1842", priority as "urgent", and add an unsupported field such as issue_summary. It may still look structured and readable, but it does not satisfy the agreed schema.
The following script uses only the Python standard library. It loads a synthetic AI response, checks that no unexpected keys are present, verifies required keys and types, tests the allowed priority values, checks the product list, and validates the callback time format. It writes a report to outputs and stops if the report already exists.
import json
import re
from pathlib import Path
AI_RESPONSE = '''{
"ticket_id": "1842",
"customer": "Mira Lee",
"priority": "urgent",
"products": ["Desktop", "Web"],
"callback_time": "15:30",
"issue_summary": "Login problem"
}'''
REQUIRED_KEYS = {
"ticket_id",
"customer",
"priority",
"products",
"callback_time",
}
ALLOWED_PRIORITIES = {"low", "medium", "high"}
TIME_PATTERN = re.compile(r"^(?:[01]\d|2[0-3]):[0-5]\d$")
errors = []
try:
data = json.loads(AI_RESPONSE)
except json.JSONDecodeError as exc:
raise SystemExit(f"Invalid JSON: {exc}")
if not isinstance(data, dict):
errors.append("Top-level value must be a JSON object.")
else:
actual_keys = set(data)
missing = REQUIRED_KEYS - actual_keys
unexpected = actual_keys - REQUIRED_KEYS
if missing:
errors.append("Missing keys: " + ", ".join(sorted(missing)))
if unexpected:
errors.append("Unexpected keys: " + ", ".join(sorted(unexpected)))
if "ticket_id" in data and not isinstance(data["ticket_id"], int):
errors.append("ticket_id must be an integer.")
if "customer" in data:
customer = data["customer"]
if not isinstance(customer, str) or not customer.strip():
errors.append("customer must be a non-empty string.")
if "priority" in data:
priority = data["priority"]
if not isinstance(priority, str) or priority not in ALLOWED_PRIORITIES:
errors.append("priority must be low, medium, or high.")
if "products" in data:
products = data["products"]
if not isinstance(products, list) or not products:
errors.append("products must be a non-empty list.")
elif not all(isinstance(item, str) and item.strip() for item in products):
errors.append("Every product must be a non-empty string.")
if "callback_time" in data:
callback = data["callback_time"]
if callback is not None:
if not isinstance(callback, str) or not TIME_PATTERN.fullmatch(callback):
errors.append("callback_time must be HH:MM or null.")
status = "PASS" if not errors else "FAIL"
lines = [f"Validation: {status}"]
lines.extend(f"- {error}" for error in errors)
output_dir = Path("outputs")
output_dir.mkdir(exist_ok=True)
output_file = output_dir / "json_validation.txt"
if output_file.exists():
raise SystemExit(f"Stop: {output_file} already exists.")
output_file.write_text("\n".join(lines) + "\n", encoding="utf-8")
print(f"Wrote {output_file}")
For the defective synthetic response, validation should fail for three reasons. ticket_id is a string instead of an integer. priority is urgent, which is outside the allowed set. issue_summary is an unexpected key. The products list and callback_time format satisfy their structural rules.
| Check | AI value | Expected | Result |
|---|---|---|---|
| ticket_id | "1842" | integer | FAIL |
| customer | "Mira Lee" | non-empty string | PASS |
| priority | "urgent" | low, medium, or high | FAIL |
| products | ["Desktop", "Web"] | non-empty string list | PASS |
| callback_time | "15:30" | HH:MM or null | PASS |
| extra keys | issue_summary | none | FAIL |
A schema failure should normally send the result back for correction rather than silently coercing it. Automatically converting "1842" to 1842 may hide the fact that the extraction instructions were not followed. Whether coercion is acceptable depends on the downstream workflow.
Structural validation is only one layer. You should also compare important fields with the original text. In this example, ticket 1842, Mira Lee, high priority, Desktop, Web, and 15:30 all appear explicitly in the source. The source does not use the word urgent, so replacing high with urgent changes the stated value rather than merely reformatting it.
A common mistake is checking only whether json.loads accepts the response. Successful parsing proves that the syntax is valid JSON, not that required keys, types, or values are correct. Another mistake is allowing arbitrary extra fields, which can make downstream code depend on information that was never part of the specification.
For larger schemas, hand-written validation code becomes harder to maintain. A dedicated schema system or validation library may be appropriate, but the same basic questions remain: which fields are required, what types are allowed, what values are permitted, and whether missing information may be represented as null.
Validation also does not establish that the original text is complete or trustworthy. If the source itself contains an incorrect ticket number or ambiguous wording, a structurally correct extraction will preserve that problem. Keep the original source available so important fields can be traced back when a validation or business rule fails.
2026-09-21 · hand-checked example · Python 3.12
The explanations and examples were written for this site. See the official sources below for the related behavior and concepts.