Skip to content
Episode 90 · Inside means nothing

Zero trust: identity over network location

Monday, 07:50: an attacker who got into a forgotten internal reports tool on Saturday has been calling other StayHub services for 41 hours, and every one of them answered, because the calls came from inside the network.

Being "inside the network" used to mean being trusted. Zero trust checks who you are on every request instead.

30 min advancedZero trust vs perimeter securityService identity and mutual TLSIAM least privilege for humans and workloadsRate limiting and DDoS mitigationLeast privilege for users, services and tokensNever trust the client
  1. Problem

    StayHub now has 30 services, staff working from hotels and home, contractors, partner integrations and cloud resources in several accounts. Who and what should be able to reach what?

  2. Why the simple system fails

    The perimeter model (VPN in, then everything internal trusts everything) means one phished laptop or one compromised service can move laterally to the payment database. Flat networks, shared admin keys and wildcard IAM policies turn small breaches into total ones.

  3. Engineering concept

    Never trust, always verify: every request is authenticated with a strong identity (user + device, or workload), authorized against least-privilege policy, and encrypted, regardless of where it comes from. Assume breach and limit blast radius.

  4. Technology

    Identity provider with MFA/SSO, identity-aware proxies, device posture checks, mTLS with short-lived workload certificates (often via a service mesh), cloud IAM roles with scoped policies, network segmentation as an extra layer, rate limiting, WAF and DDoS protection at the edge.

  5. Trade-offs

    Zero trust adds identity plumbing, certificate management, policy sprawl and some latency. Strict least privilege slows teams if granting access is painful. Done incrementally, starting with the highest-value assets, it pays off; as a big-bang project it stalls.

01

The problem

2 min
Monday, 07:50. Omar has spent an hour reading access logs.
  1. Omar · On-call SRE
    Saturday 14:50: a contractor's VPN login. Password phished, no MFA. Then code running on reports-v1, an old unpatched tool.
  2. Omar · On-call SRE
    For 41 hours it called booking's internal API (guest lists, 2 hotels) and payments' /refunds (3 refunds of €1). Every call succeeded.
  3. Lina · Senior engineer, your mentor
    You lead this one. Start with the why.
02

The big idea

4 min
Through the soft inside
The big idea

Being inside the network grants nothing: every request proves who it is and is checked against policy.

The old hotel had a guard at the front door and open corridors behind it. The modern one keeps the guard, but every door has a card reader: your card opens only the doors your job needs, and every swipe is logged.

03

How it works

5 min
Every request, person or service
  1. 1
    People: identity + device

    Staff tools sit behind an : SSO, MFA and a managed-device check per app. No network-wide VPN access.

  2. 2
    Services: workload identity

    Each service gets a short-lived certificate for its workload identity, issued and rotated by the . Identity, not IP address: anything inside can send from an allowed subnet.

  3. 3
    Protected channel

    : both sides prove who they are and the traffic is encrypted. Plain text is refused.

  4. 4
    Default-deny policy

    Default deny: nothing is allowed unless a rule says so. Payments accepts POST /charges from booking and POST /refunds from the admin console. Nothing else.

  5. 5
    Narrow cloud permissions

    Each service's role names exact actions on exact resources. No "Action": "*" like reports-v1 had on the invoice bucket.

  6. 6
    Log every decision, limit the edge

    Allowed and denied calls go to the audit log. At the edge, per caller. Each hop now needs its own proof, which stops lateral movement: hopping from one compromised machine to the next.

04

Watch a decision being made

4 min
Worked example

You plan StayHub's zero-trust rollout, Lina checks

Make sure the next foothold (a phished account, a vulnerable tool, an SSRF) reaches as little as possible, without breaking StayHub on the way.

  1. 1Map the crown jewels

    Payments, guest data, the admin console, the cloud account. List every path that reaches them today.

    Why: Lina: "You can't protect everything at once; start where a breach hurts most."

  2. 2People first

    Every staff tool behind the identity-aware proxy. Retire network-wide VPN access; decommission reports-v1.

    Why: The foothold was a password-only VPN. Biggest risk reduction per euro, no change inside services.

  3. 3Services: observe, then enforce

    Mesh in permissive mode everywhere: certificates issued, plaintext still allowed, plaintext callers listed. Then strict plus allow rules, payments first, one namespace at a time.

    yaml
    kind: PeerAuthentication
metadata: { name: strict-mtls, namespace: payments }
spec: { mtls: { mode: STRICT } }   # plaintext rejected

    Why: Observing first shows who would break before anything breaks; one namespace keeps a mistake small.

  4. 4Scope cloud roles, limit the edge

    Replace wildcards with the actions each service used in the last 90 days. Login limits: 5/min per account, 20/min per IP.

    Why: Narrow roles keep a leak narrow. Per-IP limits stay loose because hotel Wi-Fi puts ~200 guests behind one IP.

The whole thing in one line: People's access first, then service identities observed in permissive mode, default deny around payments one namespace at a time, and no wildcard roles.
05

In real life

3 min
Two starting points

Google BeyondCorp: people first

  • Publicly described moving employee access away from a privileged VPN network
  • Access to each app depends on user identity and device state

Stops phished accounts and lost laptops reaching everything.

Lyft: services first, with Envoy

  • Built Envoy, a proxy beside each service, now the basis of many service meshes
  • Identity, encryption and policy on every call without changing app code

Stops one compromised service reaching the rest.

06

What goes wrong

3 min
Strict mTLS everywhere in one go

Callers without certificates fail; the project gets rolled back.

Instead

Permissive first, empty the plaintext list, then enforce namespace by namespace with a rollback ready.

"It's internal, so it doesn't need auth"

Internal endpoints are exactly what SSRF and compromised tools reach.

Instead

Authenticate and authorize every service call; default deny.

Wildcard IAM policies to "unblock" a team

Any leak of that credential becomes full access.

Instead

Start from the actions actually used; grant temporary elevation when needed.

07

Trade-offs

3 min
OptionFits whenYou pay
Perimeter only (VPN + firewall)A tiny team, as a startOne stolen credential reaches everything
Identity-aware proxy for peopleRemote staff, contractors, sensitive toolsA solid identity provider and device management
Service mesh with mTLS + policiesMany services on KubernetesSidecars on every pod, a control plane, harder debugging
Signed tokens in app codeA few services, mixed platformsEach team implements it; inconsistencies creep in
Tuesday, 11:00. Choosing how services prove who they are.
  1. Karim · CTO
    A service mesh for all twelve services. mTLS and policies without touching app code.
  2. Omar · On-call SRE
    A sidecar beside every pod and harder debugging. I'd add signed tokens to the five payment-path services.
  3. Karim · CTO
    Then five teams do it differently, and the other seven stay open.
08

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own
  • Which service may call which: booking creates charges, admin refunds, everything else denied. An agent proposes the allow list; you approve it.
  • The rollout order for : permissive first, then strict one namespace at a time, starting with payments.
  • What lives outside the and how it authenticates: the payout VM and the legacy extranet.
Tell Claude Code
We're moving payments to strict mTLS. Plan only, no changes. List every caller of payments, inside and outside the mesh, and how you found it (config, traffic logs, code); the nightly payout job runs on a VM outside the mesh. Propose a default-deny policy allowing only booking to POST /charges and admin to POST /refunds. Rollout: permissive in the payments namespace for a week, then strict for that namespace only, on a weekday morning, with a one-line rollback.
Check before you accept
  • Claude's caller list includes jobs, VMs and old tools: that's what broke last time.
  • The policy names who is allowed; "allow from the cluster's IP range" is still the old castle.
  • The plan changes one namespace, with a time, an owner watching and the rollback command.
Red flags
  • It turns on strict mTLS everywhere because "it's just a config flag".
  • It allows all traffic from inside the VPC "to avoid breaking internal callers".
  • It gives a service a wildcard IAM policy to make a permission error go away.
09

Check yourself

2 min
Q1/3Scenario
An attacker gains code execution in StayHub's review service (inside the cluster). In which setup can they NOT call the payment service's POST /charges?
How confident are you?

Next episode: a one-line pricing change passes every test, and Saturday nights start selling at weekday prices.

Finished?

Mark it complete to update your skill map.

Next: The testing pyramid: which test catches which bug?
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Visual diagram, Analogy, Check your understanding, Practice ladder, Explain it yourself.
10

Technical deep dive

Identity between services
Worked example

You plan StayHub's zero-trust rollout, Lina checks

Make sure the next foothold (a phished account, a vulnerable tool, an SSRF) reaches as little as possible, without breaking StayHub on the way.

  1. 1Map the crown jewels

    Payments (refunds, charges), guest data (booking, passports), the admin console, and the cloud account. List every path that reaches them today.

    Why: Lina: "Good. You can't protect everything at once; start where a breach hurts most."

  2. 2People first

    Put every staff tool behind an identity-aware proxy with SSO, MFA and a device check. Retire network-wide VPN access; decommission reports-v1.

    Why: The foothold was a password-only VPN. This is the biggest risk reduction per euro, and it needs no change inside services.

  3. 3Give services identities

    Mesh in permissive mode in every namespace: services get certificates, plain text still works. Watch which connections are still plaintext.

    Why: Observing first tells you who would break before anything breaks.

  4. 4Default deny around payments

    Write allow rules for payments and booking from the observed traffic, then switch those namespaces to strict, one at a time.

    yaml
    kind: PeerAuthentication
metadata: { name: strict-mtls, namespace: payments }
spec: { mtls: { mode: STRICT } }   # plaintext connections rejected
# plus AuthorizationPolicy: booking → POST /charges, admin-console → POST /refunds

    Why: Strict mode plus default deny is what stops lateral movement. Doing it per namespace keeps a mistake small and reversible.

  5. 5Scope cloud permissions

    Replace wildcard roles with the actions each service actually used in the last 90 days (from the cloud's access logs). Services use workload identity, not static keys.

    Why: reports-v1's "Action": "*" on the invoice bucket would have let it read every invoice. Narrow roles keep a leak narrow.

  6. 6Limit and watch the edge

    Rate limits on login (5/min per account, 20/min per IP) and search (per API key); log every allow and deny decision; alert on denied calls to payments.

    ts
    // token bucket per key; 429 + Retry-After when empty
if (!(await allow(`login:acct:${email}`, 5, 5 / 60))) return tooMany();
if (!(await allow(`login:ip:${ip}`, 20, 20 / 60))) return tooMany();

    Why: Denied calls are often the first sign of someone exploring. Rate limits make guessing passwords slow and expensive.

The whole thing in one line: You protected people's access first, then gave services identities in permissive mode, enforced default deny around payments one namespace at a time, removed wildcard cloud permissions, and added limits and logs at the edge.
11

Visual diagram

Click a component

StayHub with zero-trust controls: identity at the edge for people, mTLS + policy between services, scoped IAM for cloud resources.

12

Analogy

  • A hotelOne guard at the front door onlyYour systemsCastle-and-moat network security
  • A hotelCard reader on every doorYour systemsChecking every single request
  • A hotelCard opens only doors your role needsYour systemsLeast privilege
  • A hotelCard expires at checkoutYour systemsShort-lived credentials
  • A hotelBar serves each guest a few drinks an hourYour systemsRate limiting per caller

Where the analogy breaks: hotel guests are people. In software, most callers are other programs, and they need identities and rules too.

13

Check your understanding

Q1/3Debugging
A CI deploy key leaked. Its policy is "Action": "*", "Resource": "*". Which change most reduces the blast radius of the next leak?
How confident are you?
14

Practice ladder

Step 1 · Trace

Which calls get through?

Scenario Warm-up

The payments namespace is in strict mTLS with this default-deny policy:

  • booking (service account booking) may POST /charges
  • admin-console may POST /refunds

Trace each call: allowed or denied, and by which check?

  1. booking → POST /charges
  2. booking → POST /refunds
  3. reports-v2 (inside the mesh) → GET /charges
  4. The payout job on a VM outside the mesh, over plain HTTP → POST /payouts
  5. An attacker's pod that sets the header X-Service: booking → POST /charges
Step 2 · Complete

Narrow the reports role

Code Practice

reports-v2 needs to read report files under reports/ in the invoice bucket and write finished PDFs under reports/out/. Its current role, copied from reports-v1:

{ "Effect": "Allow", "Action": "s3:*", "Resource": "arn:aws:s3:::stayhub-invoices-prod/*" }

Fill in the replacement's two statements (gaps 1 and 2).

{ "Version": "2012-10-17", "Statement": [
  { "Effect": "Allow", "Action": /* gap 1 */, "Resource": /* … */ },
  { "Effect": "Allow", "Action": /* gap 2 */, "Resource": /* … */ }
] }
Step 3 · Solve

Stop credential stuffing at login

Design Practice

Saturday night, login traffic jumps from 40 to 3,000 attempts a minute, from ~12,000 different IPs, each trying a few email/password pairs leaked from other websites. Success rate: 0.4% (people who reused passwords). Meanwhile, a hotel's Wi-Fi puts ~200 guests behind one IP on a normal evening.

Design the defences: what you limit and by what key, what happens to a guest who's limited, and what else you'd add.

Step 4 · Decide

Replace the VPN this quarter?

Decision Stretch

StayHub has 40 staff and 30 contractors on the VPN, which gives everyone network access to every internal tool. Options for this quarter:

  • A. Keep the VPN, add MFA, and split it into 3 network zones. ~2 weeks.
  • B. Identity-aware proxy for the 14 internal tools (SSO + MFA + device check per app), and switch the VPN off. ~6 weeks; 3 legacy tools need rework; contractors must enrol their devices.

Write a short decision: "We chose X over Y because…, accepting the risk of…".

15

Explain it yourself

Explain the difference between perimeter security and zero trust, using an attacker who compromises one internal service as the example.

0 words