JSON Data Validator
Abstract
Section titled “Abstract”JSON is the data format of the modern web — APIs, configs, telemetry, NoSQL stores all live in it. But JSON gives you no type safety: a misnamed field or out-of-range number sails through json.loads() and silently breaks downstream code hours later. A schema validator catches those errors at the boundary, with precise error messages pointing at the offending field. In this project you build one from scratch — type checking, constraints, nested validation, path-tracked errors, batch validation, and sample-data generation — then compare it to the production-grade libraries jsonschema and pydantic.
You will leave understanding:
- The JSON Schema specification (the subset most apps actually use).
- How to write a small recursive validator.
- Path-tracked error messages (
root.users[0].email). - The difference between schema-based and class-based validation.
- When to graduate to
jsonschema(for spec compliance) orpydantic(for type-driven validation).
Prerequisites
Section titled “Prerequisites”- Python 3.7 or above (for
dataclasses). - A text editor or IDE.
- Familiarity with JSON, dictionaries, and recursive functions.
flowchart TD
n0(["script start"])
n52["create_sample_schemas()"]
n2["main()"]
subgraph JSONDataValidator
n53["add_schema()"]
n54["batch_validate()"]
n55["create_sample_data()"]
n56["export_validation_report()"]
n57["get_schema_info()"]
n58["get_validation_statistics()"]
n59["load_schema_from_file()"]
n60["validate_data()"]
n61["validate_file()"]
n62["validate_json_string()"]
end
subgraph JSONSchema
n63["_validate_array_items()"]
n64["_validate_constraints()"]
n65["_validate_properties()"]
n66["_validate_recursive()"]
n67["_validate_required_fields()"]
n68["_validate_type()"]
n69["validate()"]
end
n54 --> n61
n61 --> n60
n62 --> n60
n63 --> n66
n65 --> n66
n66 --> n63
n66 --> n64
n66 --> n65
n66 --> n67
n66 --> n68
n69 --> n66
n0 --> n2
n2 --> n53
n2 --> n54
n2 --> n55
n2 --> n56
n2 --> n57
n2 --> n58
n2 --> n59
n2 --> n61
n2 --> n62
n2 --> n52
Getting Started
Section titled “Getting Started”Create the project
Section titled “Create the project”- Create folder
json-validator. - Inside, create
jsondatavalidator.py.
Write the code
Section titled “Write the code”JSON Validator
pch.viewSource# JSON Data Validator
import json
import sys
import os
import re
from typing import Any, Dict, List, Optional, Union, Tuple
from datetime import datetime
from pathlib import Path
class ValidationError:
def __init__(self, path: str, message: str, expected: str = None, actual: str = None):
self.path = path
self.message = message
self.expected = expected
self.actual = actual
def __str__(self):
result = f"Path: {self.path} - {self.message}"
if self.expected:
result += f" (Expected: {self.expected}"
if self.actual:
result += f", Got: {self.actual}"
result += ")"
return result
class JSONSchema:
def __init__(self, schema: Dict):
self.schema = schema
self.errors = []
def validate(self, data: Any, path: str = "root") -> Tuple[bool, List[ValidationError]]:
"""Validate data against schema"""
self.errors = []
self._validate_recursive(data, self.schema, path)
return len(self.errors) == 0, self.errors
def _validate_recursive(self, data: Any, schema: Dict, path: str):
"""Recursively validate data against schema"""
# Check type
if "type" in schema:
if not self._validate_type(data, schema["type"], path):
return
# Check required fields for objects
if isinstance(data, dict) and "required" in schema:
self._validate_required_fields(data, schema["required"], path)
# Check properties for objects
if isinstance(data, dict) and "properties" in schema:
self._validate_properties(data, schema["properties"], path)
# Check array items
if isinstance(data, list) and "items" in schema:
self._validate_array_items(data, schema["items"], path)
# Check constraints
self._validate_constraints(data, schema, path)
def _validate_type(self, data: Any, expected_type: str, path: str) -> bool:
"""Validate data type"""
type_mapping = {
"string": str,
"number": (int, float),
"integer": int,
"boolean": bool,
"array": list,
"object": dict,
"null": type(None)
}
if expected_type not in type_mapping:
self.errors.append(ValidationError(path, f"Unknown type: {expected_type}"))
return False
expected_python_type = type_mapping[expected_type]
if not isinstance(data, expected_python_type):
actual_type = type(data).__name__
self.errors.append(ValidationError(
path, "Type mismatch", expected_type, actual_type
))
return False
return True
def _validate_required_fields(self, data: Dict, required_fields: List[str], path: str):
"""Validate required fields in object"""
for field in required_fields:
if field not in data:
self.errors.append(ValidationError(
f"{path}.{field}", f"Required field '{field}' is missing"
))
def _validate_properties(self, data: Dict, properties: Dict, path: str):
"""Validate object properties"""
for key, value in data.items():
if key in properties:
self._validate_recursive(value, properties[key], f"{path}.{key}")
# Note: Additional properties are allowed by default
def _validate_array_items(self, data: List, items_schema: Dict, path: str):
"""Validate array items"""
for i, item in enumerate(data):
self._validate_recursive(item, items_schema, f"{path}[{i}]")
def _validate_constraints(self, data: Any, schema: Dict, path: str):
"""Validate additional constraints"""
# String constraints
if isinstance(data, str):
if "minLength" in schema and len(data) < schema["minLength"]:
self.errors.append(ValidationError(
path, f"String too short (min: {schema['minLength']})",
str(schema["minLength"]), str(len(data))
))
if "maxLength" in schema and len(data) > schema["maxLength"]:
self.errors.append(ValidationError(
path, f"String too long (max: {schema['maxLength']})",
str(schema["maxLength"]), str(len(data))
))
if "pattern" in schema:
if not re.match(schema["pattern"], data):
self.errors.append(ValidationError(
path, f"String does not match pattern: {schema['pattern']}"
))
# Number constraints
if isinstance(data, (int, float)):
if "minimum" in schema and data < schema["minimum"]:
self.errors.append(ValidationError(
path, f"Number too small (min: {schema['minimum']})",
str(schema["minimum"]), str(data)
))
if "maximum" in schema and data > schema["maximum"]:
self.errors.append(ValidationError(
path, f"Number too large (max: {schema['maximum']})",
str(schema["maximum"]), str(data)
))
# Array constraints
if isinstance(data, list):
if "minItems" in schema and len(data) < schema["minItems"]:
self.errors.append(ValidationError(
path, f"Array too short (min items: {schema['minItems']})",
str(schema["minItems"]), str(len(data))
))
if "maxItems" in schema and len(data) > schema["maxItems"]:
self.errors.append(ValidationError(
path, f"Array too long (max items: {schema['maxItems']})",
str(schema["maxItems"]), str(len(data))
))
# Enum constraint
if "enum" in schema:
if data not in schema["enum"]:
self.errors.append(ValidationError(
path, f"Value not in allowed enum values: {schema['enum']}",
str(schema["enum"]), str(data)
))
class JSONDataValidator:
def __init__(self):
self.schemas = {}
self.validation_results = []
def load_schema_from_file(self, schema_file: str, schema_name: str = None) -> bool:
"""Load schema from JSON file"""
try:
with open(schema_file, 'r', encoding='utf-8') as f:
schema_data = json.load(f)
name = schema_name or Path(schema_file).stem
self.schemas[name] = JSONSchema(schema_data)
return True
except (FileNotFoundError, json.JSONDecodeError, Exception) as e:
print(f"Error loading schema from {schema_file}: {e}")
return False
def add_schema(self, schema_name: str, schema_dict: Dict) -> bool:
"""Add schema from dictionary"""
try:
self.schemas[schema_name] = JSONSchema(schema_dict)
return True
except Exception as e:
print(f"Error adding schema {schema_name}: {e}")
return False
def validate_file(self, json_file: str, schema_name: str) -> Tuple[bool, List[ValidationError]]:
"""Validate JSON file against schema"""
if schema_name not in self.schemas:
error = ValidationError("", f"Schema '{schema_name}' not found")
return False, [error]
try:
with open(json_file, 'r', encoding='utf-8') as f:
data = json.load(f)
return self.validate_data(data, schema_name)
except FileNotFoundError:
error = ValidationError("", f"File '{json_file}' not found")
return False, [error]
except json.JSONDecodeError as e:
error = ValidationError("", f"Invalid JSON in file '{json_file}': {e}")
return False, [error]
except Exception as e:
error = ValidationError("", f"Error reading file '{json_file}': {e}")
return False, [error]
def validate_data(self, data: Any, schema_name: str) -> Tuple[bool, List[ValidationError]]:
"""Validate data against schema"""
if schema_name not in self.schemas:
error = ValidationError("", f"Schema '{schema_name}' not found")
return False, [error]
schema = self.schemas[schema_name]
is_valid, errors = schema.validate(data)
# Store result
result = {
'timestamp': datetime.now().isoformat(),
'schema_name': schema_name,
'is_valid': is_valid,
'error_count': len(errors),
'errors': [str(error) for error in errors]
}
self.validation_results.append(result)
return is_valid, errors
def validate_json_string(self, json_string: str, schema_name: str) -> Tuple[bool, List[ValidationError]]:
"""Validate JSON string against schema"""
try:
data = json.loads(json_string)
return self.validate_data(data, schema_name)
except json.JSONDecodeError as e:
error = ValidationError("", f"Invalid JSON string: {e}")
return False, [error]
def batch_validate(self, file_pattern: str, schema_name: str) -> Dict[str, Tuple[bool, List[ValidationError]]]:
"""Validate multiple files matching pattern"""
from glob import glob
def ask(prompt="", default=""):
"""Read a line, or fall back to `default` when nobody is there to type.
Without this the script raises EOFError the moment it runs unattended — in
a test, a scheduled job, or the build that captures this output for the
docs. The fallback is printed rather than silent, so a reader can always
tell which answers were typed and which were assumed.
"""
try:
return input(prompt).strip() or default
except EOFError:
print(f"{default} (no input available, using the default)")
return default
results = {}
files = glob(file_pattern)
if not files:
print(f"No files found matching pattern: {file_pattern}")
return results
for file_path in files:
print(f"Validating {file_path}...")
is_valid, errors = self.validate_file(file_path, schema_name)
results[file_path] = (is_valid, errors)
return results
def get_schema_info(self, schema_name: str) -> Optional[Dict]:
"""Get information about a schema"""
if schema_name not in self.schemas:
return None
schema = self.schemas[schema_name].schema
def analyze_schema(schema_part):
info = {}
if "type" in schema_part:
info["type"] = schema_part["type"]
if "required" in schema_part:
info["required_fields"] = schema_part["required"]
if "properties" in schema_part:
info["properties"] = {
prop: analyze_schema(prop_schema)
for prop, prop_schema in schema_part["properties"].items()
}
return info
return analyze_schema(schema)
def create_sample_data(self, schema_name: str) -> Optional[Dict]:
"""Create sample data that conforms to schema"""
if schema_name not in self.schemas:
return None
schema = self.schemas[schema_name].schema
def generate_sample(schema_part):
if "type" not in schema_part:
return None
data_type = schema_part["type"]
if data_type == "string":
if "enum" in schema_part:
return schema_part["enum"][0]
return "sample_string"
elif data_type == "number":
return 42.0
elif data_type == "integer":
return 42
elif data_type == "boolean":
return True
elif data_type == "array":
if "items" in schema_part:
return [generate_sample(schema_part["items"])]
return []
elif data_type == "object":
obj = {}
if "properties" in schema_part:
for prop, prop_schema in schema_part["properties"].items():
obj[prop] = generate_sample(prop_schema)
return obj
elif data_type == "null":
return None
return None
return generate_sample(schema)
def export_validation_report(self, filename: str):
"""Export validation results to file"""
try:
report = {
'generated_at': datetime.now().isoformat(),
'total_validations': len(self.validation_results),
'successful_validations': sum(1 for r in self.validation_results if r['is_valid']),
'failed_validations': sum(1 for r in self.validation_results if not r['is_valid']),
'results': self.validation_results
}
with open(filename, 'w', encoding='utf-8') as f:
json.dump(report, f, indent=2)
print(f"Validation report exported to {filename}")
except Exception as e:
print(f"Error exporting report: {e}")
def get_validation_statistics(self) -> Dict:
"""Get statistics about validation results"""
if not self.validation_results:
return {}
total = len(self.validation_results)
successful = sum(1 for r in self.validation_results if r['is_valid'])
failed = total - successful
# Schema usage
schema_usage = {}
for result in self.validation_results:
schema = result['schema_name']
schema_usage[schema] = schema_usage.get(schema, 0) + 1
# Most common errors
all_errors = []
for result in self.validation_results:
all_errors.extend(result['errors'])
return {
'total_validations': total,
'successful_validations': successful,
'failed_validations': failed,
'success_rate': (successful / total * 100) if total > 0 else 0,
'schema_usage': schema_usage,
'total_errors': len(all_errors),
'loaded_schemas': list(self.schemas.keys())
}
def create_sample_schemas():
"""Create some sample schemas for demonstration"""
schemas = {
"user": {
"type": "object",
"required": ["name", "email", "age"],
"properties": {
"name": {
"type": "string",
"minLength": 2,
"maxLength": 50
},
"email": {
"examples": ["ada@example.com"],
"type": "string",
"pattern": r"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$"
},
"age": {
"type": "integer",
"minimum": 0,
"maximum": 150
},
"phone": {
"type": "string",
"examples": ["+44 20 7946 0958"],
"pattern": r"^\+?[\d\s\-\(\)]+$"
},
"status": {
"type": "string",
"enum": ["active", "inactive", "pending"]
}
}
},
"product": {
"type": "object",
"required": ["name", "price", "category"],
"properties": {
"name": {
"type": "string",
"minLength": 1,
"maxLength": 100
},
"price": {
"type": "number",
"minimum": 0
},
"category": {
"type": "string",
"enum": ["electronics", "clothing", "books", "home", "sports"]
},
"tags": {
"type": "array",
"items": {
"type": "string"
},
"maxItems": 10
},
"in_stock": {
"type": "boolean"
}
}
},
"config": {
"type": "object",
"required": ["app_name", "version"],
"properties": {
"app_name": {
"type": "string",
"minLength": 1
},
"version": {
"type": "string",
"pattern": r"^\d+\.\d+\.\d+$"
},
"debug": {
"type": "boolean"
},
"features": {
"type": "array",
"items": {
"type": "string"
}
},
"database": {
"type": "object",
"required": ["host", "port"],
"properties": {
"host": {
"type": "string"
},
"port": {
"type": "integer",
"minimum": 1,
"maximum": 65535
},
"name": {
"type": "string"
}
}
}
}
}
}
return schemas
def main():
"""Main function to run the JSON data validator"""
validator = JSONDataValidator()
# Load sample schemas
sample_schemas = create_sample_schemas()
for name, schema in sample_schemas.items():
validator.add_schema(name, schema)
while True:
print("\n=== JSON Data Validator ===")
print("1. Validate JSON file")
print("2. Validate JSON string")
print("3. Batch validate files")
print("4. Load schema from file")
print("5. Add schema manually")
print("6. View schema info")
print("7. Generate sample data")
print("8. View validation statistics")
print("9. Export validation report")
print("10. List available schemas")
print("0. Exit")
try:
choice = ask("\nEnter your choice: ", '0').strip()
if choice == '1':
json_file = ask("Enter JSON file path: ", '0').strip()
print("\nAvailable schemas:")
for schema_name in validator.schemas.keys():
print(f" • {schema_name}")
schema_name = ask("Enter schema name: ", 'Demo').strip()
if schema_name in validator.schemas:
is_valid, errors = validator.validate_file(json_file, schema_name)
if is_valid:
print("✅ Validation successful!")
else:
print("❌ Validation failed!")
print(f"Found {len(errors)} errors:")
for error in errors:
print(f" • {error}")
else:
print("Schema not found!")
elif choice == '2':
print("Enter JSON string (end with empty line):")
json_lines = []
while True:
line = ask("", '0')
if line.strip() == "":
break
json_lines.append(line)
json_string = '\n'.join(json_lines)
print("\nAvailable schemas:")
for schema_name in validator.schemas.keys():
print(f" • {schema_name}")
schema_name = ask("Enter schema name: ", 'Demo').strip()
if schema_name in validator.schemas:
is_valid, errors = validator.validate_json_string(json_string, schema_name)
if is_valid:
print("✅ Validation successful!")
else:
print("❌ Validation failed!")
print(f"Found {len(errors)} errors:")
for error in errors:
print(f" • {error}")
else:
print("Schema not found!")
elif choice == '3':
file_pattern = ask("Enter file pattern (e.g., *.json, data/*.json): ", '0').strip()
print("\nAvailable schemas:")
for schema_name in validator.schemas.keys():
print(f" • {schema_name}")
schema_name = ask("Enter schema name: ", 'Demo').strip()
if schema_name in validator.schemas:
results = validator.batch_validate(file_pattern, schema_name)
print(f"\nBatch validation results:")
for file_path, (is_valid, errors) in results.items():
status = "✅" if is_valid else "❌"
print(f"{status} {file_path}: {len(errors)} errors")
if errors and len(errors) <= 3: # Show first few errors
for error in errors[:3]:
print(f" • {error}")
else:
print("Schema not found!")
elif choice == '4':
schema_file = ask("Enter schema file path: ", '0').strip()
schema_name = ask("Enter schema name (optional): ", '0').strip()
if validator.load_schema_from_file(schema_file, schema_name or None):
print("Schema loaded successfully!")
else:
print("Failed to load schema.")
elif choice == '5':
schema_name = ask("Enter schema name: ", 'Demo').strip()
print("Enter schema JSON (end with empty line):")
schema_lines = []
while True:
line = ask("", '0')
if line.strip() == "":
break
schema_lines.append(line)
schema_string = '\n'.join(schema_lines)
try:
schema_dict = json.loads(schema_string)
if validator.add_schema(schema_name, schema_dict):
print("Schema added successfully!")
else:
print("Failed to add schema.")
except json.JSONDecodeError as e:
print(f"Invalid JSON schema: {e}")
elif choice == '6':
print("\nAvailable schemas:")
for schema_name in validator.schemas.keys():
print(f" • {schema_name}")
schema_name = ask("Enter schema name: ", 'Demo').strip()
info = validator.get_schema_info(schema_name)
if info:
print(f"\nSchema '{schema_name}' information:")
print(json.dumps(info, indent=2))
else:
print("Schema not found!")
elif choice == '7':
print("\nAvailable schemas:")
for schema_name in validator.schemas.keys():
print(f" • {schema_name}")
schema_name = ask("Enter schema name: ", 'Demo').strip()
sample_data = validator.create_sample_data(schema_name)
if sample_data is not None:
print(f"\nSample data for '{schema_name}' schema:")
print(json.dumps(sample_data, indent=2))
else:
print("Schema not found or couldn't generate sample data!")
elif choice == '8':
stats = validator.get_validation_statistics()
if stats:
print("\n=== Validation Statistics ===")
print(f"Total validations: {stats['total_validations']}")
print(f"Successful: {stats['successful_validations']}")
print(f"Failed: {stats['failed_validations']}")
print(f"Success rate: {stats['success_rate']:.1f}%")
print(f"Total errors: {stats['total_errors']}")
if stats['schema_usage']:
print("\nSchema usage:")
for schema, count in stats['schema_usage'].items():
print(f" {schema}: {count} validations")
print(f"\nLoaded schemas: {', '.join(stats['loaded_schemas'])}")
else:
print("No validation statistics available.")
elif choice == '9':
filename = ask("Enter report filename (e.g., validation_report.json): ", 'demo.txt').strip()
if not filename:
filename = f"validation_report_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json"
validator.export_validation_report(filename)
elif choice == '10':
print("\nAvailable schemas:")
if validator.schemas:
for schema_name in validator.schemas.keys():
schema_info = validator.get_schema_info(schema_name)
schema_type = schema_info.get('type', 'unknown') if schema_info else 'unknown'
print(f" • {schema_name} (type: {schema_type})")
else:
print(" No schemas loaded.")
elif choice == '0':
print("Thank you for using JSON Data Validator!")
break
else:
print("Invalid choice. Please try again.")
except KeyboardInterrupt:
print("\n\nGoodbye!")
break
except Exception as e:
print(f"An error occurred: {e}")
# --------------------------------------------------------------------------
# The functional surface the documentation page teaches. The class above is
# the application; these are the three verbs on top of it, and they exist
# here so that a snippet copied off the page runs against this file.
# --------------------------------------------------------------------------
def sample(schema: dict):
"""Build one example document that satisfies `schema`.
Useful for two things: showing a user what shape is expected, and giving
a test suite a valid starting point to then break in one specific way.
Everything it returns is the least interesting legal value -- the minimum
for a number, the first enum member -- because a sample is a shape, not
a fixture.
"""
# A generator cannot invent a string matching an arbitrary regular
# expression -- that needs a regex-reversing library, and the general
# problem is genuinely hard. So the schema is asked first: `default` and
# `examples` are standard JSON Schema keywords and exist for exactly this.
if "default" in schema:
return schema["default"]
if schema.get("examples"):
return schema["examples"][0]
kind = schema.get("type")
if kind == "string":
if schema.get("enum"):
return schema["enum"][0]
# Without an example the best available guess is a placeholder padded
# to the minimum length. It will still fail any `pattern` constraint,
# which is why the demo checks its own output rather than assuming.
return "example".ljust(schema.get("minLength", 0), "x")
if kind == "integer":
return int(schema.get("minimum", 0))
if kind == "number":
return float(schema.get("minimum", 0))
if kind == "boolean":
return True
if kind == "array":
return [sample(schema["items"])] if "items" in schema else []
if kind == "object":
return {key: sample(value)
for key, value in schema.get("properties", {}).items()}
return None
def batch(pattern: str, schema: dict, validator=None):
"""Validate every file matching a glob, returning (path, ok, errors).
The return value matters more than the printing. A batch validator that
only prints cannot be used by anything else -- not a test, not a CI step,
not a report -- so it returns the results and prints as a courtesy.
"""
from pathlib import Path
validator = validator or JSONSchema(schema)
results = []
for path in sorted(Path().glob(pattern)):
try:
data = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
results.append((str(path), False,
[ValidationError(str(path), f"not JSON: {exc}")]))
print(f" BAD {path} (not valid JSON: {exc.msg} "
f"at line {exc.lineno})")
continue
ok, errors = validator.validate(data)
results.append((str(path), ok, errors))
print(f" {'OK ' if ok else 'BAD'} {path} ({len(errors)} error(s))")
return results
# `pydantic` does the same job by declaring the shape as a type. Importing it
# lazily keeps this file runnable without it -- the point of showing the
# alternative is lost if the file refuses to start when it is absent.
def build_user_model():
"""The pydantic equivalent of the user schema, or None if unavailable."""
try:
from pydantic import BaseModel, Field
except ImportError:
return None
class User(BaseModel):
name: str = Field(min_length=2, max_length=50)
# `EmailStr` needs the email-validator package; a pattern keeps this
# to one dependency and is honest about being a weaker check.
email: str = Field(pattern=r"^[^@\s]+@[^@\s]+\.[^@\s]+$")
age: int = Field(ge=0, le=150)
return User
User = build_user_model()
def demo():
"""Validate four documents against the user schema and report.
Three of them are wrong in different ways, because a validator that has
only ever seen valid input has not been tested. The point of the run is
the error messages, not the pass.
"""
schema = create_sample_schemas()["user"]
validator = JSONSchema(schema)
print("=== JSON Data Validator ===\n")
print("a document the schema would accept, generated from the schema:")
print(" " + json.dumps(sample(schema)) + "\n")
documents = {
"good.json": {"name": "Ada Lovelace", "email": "ada@example.com",
"age": 36},
"bad-type.json": {"name": "Ada", "email": "ada@example.com",
"age": "thirty-six"},
"missing.json": {"name": "A", "email": "not-an-email"},
}
for name, document in documents.items():
with open(name, "w", encoding="utf-8") as handle:
json.dump(document, handle)
with open("broken.json", "w", encoding="utf-8") as handle:
handle.write('{"name": "unterminated')
print("batch validating *.json:")
results = batch("*.json", schema, validator)
print()
for path, ok, errors in results:
if ok:
continue
print(f"{path}:")
for error in errors:
print(f" {error}")
passed = sum(1 for _, ok, _ in results if ok)
print(f"\n{passed} of {len(results)} documents valid")
if User is not None:
print("\nthe same rules as a pydantic model:")
for name, document in documents.items():
try:
User.model_validate(document)
print(f" {name:16} accepted")
except Exception as exc:
count = len(getattr(exc, "errors", lambda: [])())
print(f" {name:16} rejected, {count} error(s)")
print("\npydantic reports the same failures from a class declaration")
print("rather than a schema dict. Which one to use depends on whether")
print("the schema has to be data -- shared with another language, or")
print("loaded at runtime -- or can be code.")
else:
print("\npydantic is not installed, so that comparison was skipped.")
if __name__ == "__main__":
if "--menu" in sys.argv:
main()
else:
# The interactive menu is opt-in. Run unattended it answered "0" and
# exited, so the captured transcript showed a menu and nothing else.
demo()Run it
Section titled “Run it”python jsondatavalidator.py1. Validate file
2. Validate string
3. Batch validate
4. Generate sample
5. Statistics
6. Quit
> 1
Schema: user
File: data.json
✅ ValidA Tiny JSON Schema Example
Section titled “A Tiny JSON Schema Example”{
"type": "object",
"required": ["name", "email", "age"],
"properties": {
"name": {"type": "string", "minLength": 2, "maxLength": 50},
"email": {"type": "string", "pattern": "^.+@.+\\..+$"},
"age": {"type": "integer", "minimum": 0, "maximum": 150},
"status": {"type": "string", "enum": ["active", "inactive", "pending"]},
"tags": {"type": "array", "items": {"type": "string"}, "maxItems": 10}
}
}Matching data:
{"name": "Madhur", "email": "m@example.com", "age": 30, "status": "active"}What it produces
Section titled “What it produces”Running the file exactly as it ships takes 0.5 s and prints:
=== JSON Data Validator ===
a document the schema would accept, generated from the schema:
{"name": "example", "email": "ada@example.com", "age": 0, "phone": "+44 20 7946 0958", "status": "active"}
batch validating *.json:
BAD bad-type.json (1 error(s))
BAD broken.json (not valid JSON: Unterminated string starting at at line 1)
OK good.json (0 error(s))
BAD missing.json (3 error(s))
bad-type.json:
Path: root.age - Type mismatch (Expected: integer, Got: str)
broken.json:
Path: broken.json - not JSON: Unterminated string starting at: line 1 column 10 (char 9)
missing.json:
Path: root.age - Required field 'age' is missing
Path: root.name - String too short (min: 2) (Expected: 2, Got: 1)
Path: root.email - String does not match pattern: ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$
...The first 20 of 31 lines are shown; the run continues past this point.
Step-by-Step Explanation
Section titled “Step-by-Step Explanation”1. The error type
Section titled “1. The error type”from dataclasses import dataclass
@dataclass
class ValidationError:
path: str # e.g. "root.users[0].email"
message: str # e.g. "expected string, got int"
expected: str = ""
actual: str = ""
def __str__(self):
return f"{self.path}: {self.message}" + (
f" (expected {self.expected}, got {self.actual})"
if self.expected else "")Path-tracked errors are the single biggest UX improvement over a plain True/False validator. Users learn what is wrong where.
2. The validator
Section titled “2. The validator”import re
TYPE_MAP = {
"string": str, "integer": int, "number": (int, float),
"boolean": bool, "array": list, "object": dict, "null": type(None),
}
def validate(data, schema, path="root") -> list[ValidationError]:
errors = []
expected_type = schema.get("type")
if expected_type:
py_type = TYPE_MAP.get(expected_type)
if py_type and not isinstance(data, py_type):
errors.append(ValidationError(path, "wrong type",
expected=expected_type,
actual=type(data).__name__))
return errors # type mismatch — skip further checks
if expected_type == "string":
if "minLength" in schema and len(data) < schema["minLength"]:
errors.append(ValidationError(path, f"shorter than {schema['minLength']}"))
if "maxLength" in schema and len(data) > schema["maxLength"]:
errors.append(ValidationError(path, f"longer than {schema['maxLength']}"))
if "pattern" in schema and not re.match(schema["pattern"], data):
errors.append(ValidationError(path, f"does not match pattern {schema['pattern']}"))
if "enum" in schema and data not in schema["enum"]:
errors.append(ValidationError(path,
f"not in {schema['enum']}", actual=str(data)))
elif expected_type in ("integer", "number"):
if "minimum" in schema and data < schema["minimum"]:
errors.append(ValidationError(path, f"less than {schema['minimum']}"))
if "maximum" in schema and data > schema["maximum"]:
errors.append(ValidationError(path, f"greater than {schema['maximum']}"))
elif expected_type == "array":
if "minItems" in schema and len(data) < schema["minItems"]:
errors.append(ValidationError(path, f"fewer than {schema['minItems']} items"))
if "maxItems" in schema and len(data) > schema["maxItems"]:
errors.append(ValidationError(path, f"more than {schema['maxItems']} items"))
if "uniqueItems" in schema and len(set(map(repr, data))) != len(data):
errors.append(ValidationError(path, "items not unique"))
if "items" in schema:
for i, item in enumerate(data):
errors += validate(item, schema["items"], f"{path}[{i}]")
elif expected_type == "object":
for key in schema.get("required", []):
if key not in data:
errors.append(ValidationError(path, f"missing required '{key}'"))
for key, sub in schema.get("properties", {}).items():
if key in data:
errors += validate(data[key], sub, f"{path}.{key}")
if schema.get("additionalProperties") is False:
extras = set(data) - set(schema.get("properties", {}))
for k in extras:
errors.append(ValidationError(path, f"unexpected property '{k}'"))
return errorsTwo key design points:
- One recursive function handles every level. Nested schemas validate themselves naturally.
- Errors accumulate rather than short-circuiting. Users see all problems at once.
3. Validate a file
Section titled “3. Validate a file”import json
from pathlib import Path
def validate_file(path: str, schema: dict):
data = json.loads(Path(path).read_text(encoding="utf-8"))
errors = validate(data, schema)
return (not errors), errors
ok, errs = validate_file("user.json", user_schema)
print("Valid!" if ok else "\n".join(str(e) for e in errs))4. Batch validate with glob patterns
Section titled “4. Batch validate with glob patterns”def batch(pattern: str, schema: dict):
for p in Path().glob(pattern):
ok, errs = validate_file(str(p), schema)
print(f"{'✅' if ok else '❌'} {p} ({len(errs)} errors)")
batch("data/*.json", user_schema)5. Generate sample data
Section titled “5. Generate sample data”Useful for fixtures and tests:
def sample(schema: dict):
t = schema.get("type")
if t == "string": return schema.get("enum", ["example"])[0]
if t == "integer": return schema.get("minimum", 0)
if t == "number": return float(schema.get("minimum", 0))
if t == "boolean": return True
if t == "array": return [sample(schema["items"])] if "items" in schema else []
if t == "object":
return {k: sample(v) for k, v in schema.get("properties", {}).items()}
return NoneRun on a schema → get a fully-typed valid JSON example for documentation or tests.
Common Mistakes
Section titled “Common Mistakes”| Problem | Cause | Fix |
|---|---|---|
True passes "type": "integer" check | Python’s bool is a subclass of int | Check type(x) is int, not isinstance |
| Pattern always matches | Used re.match for partial matches | Use re.fullmatch for full-string match |
| Error message says “root” only | Did not pass path down | Always include f"{path}.{key}" in recursive call |
| Numbers wrongly rejected | Forgot int + float for "number" | TYPE_MAP["number"] = (int, float) |
Nested additionalProperties not enforced | Only checked at top level | Recurse properly |
| Schema author typo silently ignored | No schema-of-schemas check | Validate the schema itself before using |
Comparison to Production Libraries
Section titled “Comparison to Production Libraries”jsonschema (spec-compliant)
Section titled “jsonschema (spec-compliant)”pip install jsonschemafrom jsonschema import validate, ValidationError
try:
validate(instance=data, schema=schema)
except ValidationError as e:
print(e.message, e.absolute_path)Handles full Draft 2020-12 spec: $ref, oneOf, anyOf, allOf, conditional schemas, format checks, custom keywords.
pydantic (type-driven)
Section titled “pydantic (type-driven)”pip install pydanticfrom pydantic import BaseModel, Field, EmailStr
class User(BaseModel):
name: str = Field(min_length=2, max_length=50)
email: EmailStr
age: int = Field(ge=0, le=150)
User.model_validate({"name": "M", "email": "m@x.com", "age": 30})
# raises ValidationError with errors at .errors()Schemas are Python classes. Faster, friendlier, IDE-autocomplete-aware. Used by FastAPI, LangChain, and most modern Python web stacks.
When to use which
Section titled “When to use which”| Use | Tool |
|---|---|
| Quick one-off check | Build it yourself (this project) |
| Full JSON Schema compliance | jsonschema |
| Pythonic API definitions | pydantic |
| OpenAPI auto-generation | pydantic + FastAPI |
| Schema generation for non-Python consumers | jsonschema (export from pydantic) |
Variations to Try
Section titled “Variations to Try”1. Custom keywords
Section titled “1. Custom keywords”Add "isEmail" or "isUUID" as schema-level shortcuts that map to specific patterns.
2. $ref resolution
Section titled “2. $ref resolution”Lets schemas reference each other:
{"address": {"$ref": "#/definitions/address"}}3. oneOf / anyOf / allOf
Section titled “3. oneOf / anyOf / allOf”Compose schemas. Email is “string AND matches email pattern AND length ≤ 254”.
4. JSON Schema → markdown docs
Section titled “4. JSON Schema → markdown docs”Walk the schema and generate human-readable documentation for an API.
5. CLI tool
Section titled “5. CLI tool”validator schema.json data.json
validator --watch schema.json incoming/*.json6. Web service
Section titled “6. Web service”Flask endpoint that accepts schema + data and returns {"valid": bool, "errors": [...]}. See Basic Web Server.
7. Live validation in a GUI
Section titled “7. Live validation in a GUI”Tkinter with a JSON text area and a schema dropdown — errors highlight in real time.
8. Streaming validation
Section titled “8. Streaming validation”Validate huge JSON Lines files line-by-line with ijson to keep memory bounded.
9. Schema inference
Section titled “9. Schema inference”Read a sample of valid JSON, infer a permissive schema from it. Useful for legacy systems with no docs.
10. Mock data generation
Section titled “10. Mock data generation”sample(schema) already does the basic case. Add random for varied output, plus locale-aware names/emails via Faker.
Real-World Applications
Section titled “Real-World Applications”- API request/response validation — at every layer of a microservices system.
- Config-file validation — fail fast on bad YAML/JSON at startup.
- ETL pipelines — reject malformed records before they corrupt downstream.
- CI / data-quality — automated checks that incoming data matches contract.
- Documentation generation — schemas become user-facing docs automatically.
- OpenAPI / Swagger — REST APIs document themselves via JSON Schema.
Educational Value
Section titled “Educational Value”- Recursive algorithms — schema and data are both trees; you walk them in parallel.
- Error reporting UX — path tracking is the difference between debug joy and debug despair.
- Standards (JSON Schema spec) — interoperability with thousands of tools.
- Library evaluation — knowing when “build it yourself” beats “pip install” and vice versa.
- Code generation — from schema to sample, from schema to docs, from schema to types.
Next Steps
Section titled “Next Steps”- Add
additionalProperties: falseenforcement throughout. - Implement
$reffor cross-schema references. - Add
oneOf/anyOfsupport for sum types. - Compare your output to
jsonschemaon the same data. - Re-implement using
pydanticclasses and feel the speed. - Wrap with a CLI or web frontend.
Conclusion
Section titled “Conclusion”You built a recursive JSON Schema validator that catches type errors, constraint violations, and missing fields with precise error messages. The same shape (data tree + schema tree + recursive walk) underlies every validator in every web framework. From here, switching to jsonschema or pydantic is incremental — they speak the same language; they just speak it faster and more completely. Full source on GitHub. Find more data-tooling projects on Python Central Hub.
Pitfalls
Section titled “Pitfalls”- A validator that has only seen valid input has not been tested. The measured run deliberately validates four documents and three of them fail — a wrong type, a missing required field with a too-short name and a bad email, and a file that is not JSON at all.
- Malformed JSON is a different failure from invalid JSON.
json.loadsraisesJSONDecodeErrorbefore any schema is consulted, so it needs its own branch.broken.jsonabove reports “Unterminated string starting at: line 1 column 10”, which is a parser error, not a validation error. - The sample generator could not satisfy its own schema. It handled
type,enumandminimumand ignoredpattern, so the document it produced was rejected by the very schema it was built from. That is not laziness: inverting an arbitrary regular expression is genuinely hard. The fix is the standardexampleskeyword, which is what it is for. - Reporting one error at a time. Collecting every failure — 3 for
missing.json— lets a user fix them in one pass instead of one round trip per mistake. isinstance(True, int)isTruein Python. A validator whose"integer"check is a bareisinstance(value, int)acceptstrueas a number. Any type table over JSON has to special-casebool.
- Measured: 4 documents, 1 valid, and both the schema validator and the pydantic model agree on the error counts — 1 for the wrong type, 3 for the incomplete record.
- Errors carry a path (
root.age), an expectation and what was actually found. An error message without a path is unusable on nested data. sample()readsdefaultandexamplesfirst, because those are the only general answer to apatternconstraint.- Schema-as-data versus schema-as-code is the real choice between JSON Schema and pydantic: one can be shared with another language or loaded at runtime, the other gets types and editor support.
-
The sample generator produced a document its own schema rejected. What was it unable to handle?
pch.quizShowAnswer
B — The `pattern` keyword — generating a string that matches an arbitrary regular expression is a much harder problem than checking one
-
Why does malformed JSON need a separate branch from invalid JSON?
pch.quizShowAnswer
B — json.loads raises before a schema is ever consulted — the file is not a document yet, so there is nothing to validate against
-
A validator checks integers with isinstance(value, int). What slips through?
pch.quizShowAnswer
B — Booleans — bool is a subclass of int in Python, so true validates as an integer unless the check excludes it explicitly
Try it yourself
Section titled “Try it yourself”pch.coffeeTagline
pch.coffeeCtapch.feedbackHeading
pch.feedbackSubheading