Part 6A — API gateway and perimeter security
Part 6 taught you to exploit and fix individual bug classes inside an application. This short Part is the layer that sits in front of every application: how a request's payload should be validated before it ever reaches your code, how rate limiting actually works and gets bypassed, what a WAF does and doesn't stop, how the WAF category evolved into WAAP, and how to walk the OWASP API Security Top 10 end to end and point at the real control for each item — with real breaches as the evidence for why each layer exists.
Chapter 40A — API payload security in depth
In one sentence
Every API request is a payload an attacker fully controls down to the byte, so the first and strongest defence is a positive security model — validate the payload against a strict schema and reject anything that doesn't match, rather than trying to detect the infinite ways it could be wrong.
Where we are
Part 6 exploited specific parsers (XML in Ch 32, deserializers in Ch 37, templates in Ch 36). This chapter is the discipline that catches most of those classes at the door, before they reach a parser at all: what a payload validation gate should check, and the payload-shaped attacks that are easy to miss because they don't look like "injection" in the classic sense.
How it works
Positive vs negative validation — the same distinction as WAF rules (Ch 40C), applied at the application boundary:
NEGATIVE (blocklist) try to detect and reject KNOWN-BAD patterns.
Always incomplete -- you're enumerating an infinite attack surface.
POSITIVE (allowlist) define EXACTLY what a valid payload looks like
(a JSON Schema / OpenAPI schema / protobuf definition) and reject
ANYTHING that doesn't match -- type, format, range, required fields,
additional-properties. Finite, enforceable, and it fails closed.
Enforce the schema at the EDGE (API gateway or a validation middleware
that runs before any business logic), not deep inside a handler where
it's easy to forget on a new endpoint.
What a payload validation gate should check, beyond "is this valid JSON":
[ ] SCHEMA CONFORMANCE every field's type, format (email, uuid, date-
time), and allowed values; `additionalProperties: false` so unknown
fields are REJECTED, not silently ignored (this is what stops mass
assignment/overposting at the gate, before it ever reaches an ORM
that might blindly bind the whole body to a model -- G1 Ch 45).
[ ] SIZE AND SHAPE LIMITS max body size, max array length, max string
length, max object nesting depth, max number of distinct keys.
(Ch 28's DoS-input limits, applied specifically to the payload
parser itself -- a deeply nested JSON object is the JSON analogue
of Ch 32's XML entity-expansion bomb.)
[ ] TYPE COERCION / CONFUSION many frameworks silently coerce types:
a "price" field expecting a number that instead receives the STRING
"0" or the ARRAY ["0","OR","1=1"] can flow into a database driver
or a comparison in ways the developer never tested. Reject the
WRONG TYPE outright; never coerce silently on a security-relevant
field.
[ ] DUPLICATE KEYS `{"role":"user","role":"admin"}` -- the JSON spec
doesn't define which duplicate wins, and different parsers
(the gateway vs the app, or two different libraries in the same
app) can pick DIFFERENT ones. This is a payload-level PARSER
DIFFERENTIAL, structurally the same class of bug as Ch 30's request
smuggling (two components disagreeing about what a message means).
Reject payloads with duplicate keys rather than silently picking
one.
[ ] NUMERIC PRECISION a huge integer or a value with more precision
than the backend's numeric type can hold can silently overflow or
truncate (`"quantity": 99999999999999999999`) -- validate numeric
RANGES explicitly, don't trust "it's a number" to mean "it's a safe
number."
[ ] CONTENT-TYPE ENFORCEMENT if an endpoint expects
`application/json`, REJECT requests with any other declared or
sniffed content type. An endpoint that "helpfully" accepts JSON,
XML, AND form-encoded bodies on the same route multiplies its
attack surface (an XXE payload smuggled in as XML to an endpoint
everyone assumed was JSON-only) and, per Ch 40C, gives a WAF tuned
for one format nothing to inspect in the other.
[ ] CHARACTER-SET AND NORMALISATION reject control characters and
null bytes in string fields outright; normalise Unicode (NFC)
BEFORE validating, never after (G1 Ch 7) -- the same principle,
applied to every string field in the payload, not just usernames.
[ ] UNSAFE CONSUMPTION OF THE RESPONSE payload security isn't only
about what you ACCEPT -- an app that calls a third-party API and
renders a field from its JSON response (a city name from a
geolocation API, a product title from a partner feed) without the
SAME output-encoding discipline as user input (G1 Ch 43) has just
made a "trusted" upstream payload into a stored-XSS vector. Treat
every external API response as untrusted input too (this is
OWASP's API10:2023, Chapter 40E).
Where this differs by transport:
REST/JSON JSON Schema or your framework's request-validation layer
(Pydantic, Zod, class-validator, a JSON Schema middleware
at the gateway) -- reject on the first schema violation,
return a generic 400, don't echo the offending value back
verbatim (avoids reflecting an XSS/injection payload into
an error page).
XML/SOAP the schema IS the XML Schema (XSD) -- validate against it
with DTDs disabled (Ch 32); a payload that doesn't
validate against the XSD is rejected before any entity
resolution is even attempted.
GraphQL the "schema" is the GraphQL schema itself, but that alone
doesn't bound COST -- pair it with query depth/complexity
limits and disable introspection in production (Ch 40E's
API4/API9 discussion).
gRPC/protobuf the .proto definition IS the schema and is strongly
typed by design -- the remaining risks are message-size
limits (Part 5 Ch 26), unknown-field handling, and any use
of `google.protobuf.Any` (a dynamic type field), which
reintroduces a type-confusion surface if the receiving
code doesn't strictly validate which concrete type it
actually got.
Worked example
Mass assignment through a payload gate that validated presence but not shape — and the fix.
SecureShop's `PATCH /api/users/me` endpoint validates that the JSON
body is well-formed and that `name` and `bio` are strings -- but the
schema has no `additionalProperties: false`, so it silently accepts
(and an underdocumented internal helper blindly merges) whatever else
is in the body.
ATTACK:
PATCH /api/users/me
{"name": "Alice", "bio": "hi", "role": "admin", "tenant_id": "OTHER"}
The API doesn't reference `role` or `tenant_id` in its OWN handler
code, but a generic `Object.assign(user, req.body)` -- style update
helper (the exact shape of the historic Rails mass-assignment class of
bug) writes BOTH fields to the record anyway. The account is now an
admin in a DIFFERENT tenant's context.
THIS IS NOT HYPOTHETICAL: in March 2012, a security researcher
demonstrated the identical class of bug against GitHub's own Ruby on
Rails codebase -- Rails' mass-assignment default at the time bound an
ENTIRE incoming params hash to a model unless explicitly restricted,
and the researcher used it to add his own SSH key to the `rails/rails`
organisation itself, proving push access to the official repository.
The fix that followed (`strong_parameters`, now Rails' default) is
exactly the "reject unknown fields" rule above, built into the
framework.
FIX:
- schema: `additionalProperties: false`, only `name` and `bio`
listed -- the gateway/validation layer now REJECTS the payload
above with a 400 before it reaches the update helper at all.
- defence in depth: the update helper itself uses an explicit
allowlist of bindable fields (a DTO, G1 Ch 45), not a generic
merge -- so even a schema gap elsewhere doesn't reopen this path.
- `role` and `tenant_id` are never client-settable fields on ANY
endpoint; they're derived server-side from the authenticated
principal (Part 5 Ch 24's token claims), never accepted as input.
Real-world scenario 2: feature composition creating an unintended payload
Facebook disclosed in September 2018 that a chain of THREE separate,
individually-reasonable features combined to expose access tokens for
roughly 50 million accounts. In their own published account of the
incident:
1. The "View As" feature (letting a user preview their profile as
seen by someone else) was supposed to be READ-ONLY.
2. A video-uploader component, when rendered inside "View As" mode,
incorrectly generated a fully-privileged access token FOR THE
PROFILE BEING VIEWED AS, rather than a restricted, view-only
context.
3. A separate "Happy Birthday" post-composer bug caused that
wrongly-scoped token to be exposed in the page's own payload,
where an attacker automating requests against the "View As"
flow could harvest it at scale.
NONE of the three components was independently a classic injection
or auth bug -- each was a case of a RESPONSE PAYLOAD carrying more
than its calling context should ever have received (this chapter's
API3:2023 "broken object property level authorization," Ch 40E). No
single code review looking at any ONE of the three features in
isolation would have caught it; the bug only existed in the
COMPOSITION.
LESSON FOR THIS CHAPTER: schema-based, POSITIVE validation of
RESPONSE shape (not just request shape) per calling CONTEXT would
have made this class of bug structurally harder -- if "View As" mode
has its own strict response schema that simply has no FIELD capable
of carrying a privileged token, a bug in an unrelated component
can't smuggle one through it. This is why Ch 40A's schema discipline
should be applied to what your API RETURNS, not only what it accepts.
Real-world scenario 3: payload shape enabling mass, automated scraping
In January 2021, following the deplatforming of the social network
Parler, security researchers and archivists were able to
systematically download an estimated tens of terabytes of the
platform's content -- including posts, videos, and their EMBEDDED
METADATA (precise GPS coordinates in many cases) -- by walking
SEQUENTIAL, PREDICTABLE numeric post IDs through Parler's own API and
pulling the FULL response payload for each, including metadata fields
that a normal web client would never display and that a properly
scoped API response should never have included at all. Notably, even
posts users had DELETED remained fetchable through the API for a
period, because deletion had not actually removed the underlying
payload the API continued to serve.
TWO PAYLOAD-SECURITY LESSONS, DISTINCT FROM THE ACCESS-CONTROL
LESSON (this is also a textbook API1/API9 case, Ch 40E):
- the RESPONSE payload included far more than a client needed
(precise geolocation metadata embedded in media) -- Ch 40A's
"shape the response to exactly the fields the caller needs"
principle, not just "does the caller have permission to see this
object at all."
- SEQUENTIAL, PREDICTABLE identifiers made systematic, automated
enumeration of the ENTIRE payload space trivial -- opaque,
random identifiers (UUIDv4, G1 Ch 45) don't fix a missing
authorization check on their own, but they remove the "just
count upward" enumeration path this incident relied on, and they
combine with Ch 40B's rate limiting to make bulk harvesting
materially harder rather than a simple incrementing loop.
Practice (60 min)
- Take one real endpoint. Write a strict JSON Schema for its body with
additionalProperties: false, explicit types, and range limits on every numeric field. Confirm a payload with one extra field, one wrong type, and one out-of-range number are all rejected with a 400. - Send a payload with a duplicate JSON key to your app and to a second
parser/library (or
jq) — check whether they agree on which value "wins." If they don't, that's a live parser-differential risk. - Build a deeply-nested JSON payload (1,000+ levels) and a very long array; confirm your gate rejects both by DEPTH/SIZE before any application code runs, timing the rejection to confirm it's fast (not itself a DoS vector).
- Find one place in your codebase where a third-party API's response is rendered without output encoding (Ch 40E's API10). Fix it with the same context-aware encoding rule you'd apply to user input.
- Add schema validation as a CI check (Ch 58): a contract test that fails the build if a handler accepts a field not declared in its documented schema.
Common mistakes
- Validating presence, not shape. "Is
namethere?" is not "isnamea string, under 100 characters, with no control characters." - No
additionalProperties: false. The single highest-leverage schema setting against mass assignment; most frameworks default it OFF. - Coercing types silently. A permissive parser that turns
"5"into5for your convenience does the same for an attacker's crafted value. - One endpoint, multiple accepted content types "for flexibility." Multiplies attack surface and defeats content-type-scoped defences.
- Trusting third-party API responses as safe. They're still attacker- reachable input if the third party (or anything between you and it) is compromised or manipulable.
- Echoing the rejected payload back in the error response. Turns your validation error page into a reflection point for whatever was rejected.
Check yourself
- What's the difference between positive and negative payload validation, and why is positive preferred at the API boundary?
- What does
additionalProperties: falseprevent, and what historic, real-world vulnerability class does it directly close? - Why are duplicate JSON keys a security concern, structurally similar to which other bug class in this guide?
- Name four payload-shape limits worth enforcing beyond basic type-checking.
- What is "unsafe consumption" of an API response, and give a concrete example of it becoming an XSS vector.
(Answers: Appendix G.)
Further reading
- Standard: JSON Schema (
json-schema.org); the OpenAPI Specification's schema-validation tooling (openapi-generator,express-openapi-validator, and similar per-language validators). - History: Egor Homakov's 2012 GitHub mass-assignment disclosure write-ups,
and the Rails
strong_parameterschangelog that followed it. - Incident: Facebook's own September 2018 security-update post on the "View As" token-exposure vulnerability (search "Facebook Security Update About 50 Million Accounts", 2018) — a detailed vendor-published root-cause account of a feature-composition bug.
- Incident: contemporary (January 2021) reporting on the Parler data scraping (e.g. Wired, Ars Technica, and Gizmodo's technical write-ups on the sequential-ID and metadata-exposure mechanics) — read more than one source, as details were actively being reconstructed by outside researchers in real time.
- Cheat sheet: OWASP "Mass Assignment Cheat Sheet" and "REST Security Cheat Sheet" (input-validation sections).
- Tool:
zod/Pydantic/class-validatordocs for schema-first request validation in the language you use.