Consumer-Driven Contract Testing

Sam Hogarth

“Quality cannot be inspected into a product or service; it must be built into it.”
W Edwards Deming

CD Requires Fast Feedback

  • ✔️ Independent deployability
  • ✔️ Automation
  • ✔️ Shifting feedback left
  • ✔️ Collaboration over handoffs

End to End

Challenges:

  • Speed of feedback
  • Reliability of feedback
  • Managing unowned dependencies

"Any advantage you gain by talking to the real system is overwhelmed by the need to stamp out non-determinism"
Martin Fowler
"Push tests as low as they can go for the highest return in investment and quickest feedback"
Janet Gregory and Lisa Crispin

Decomposing what we do:

  1. Connectivity - Mocks and Smoke Testing
  2. Conversation - Contract Testing
  3. Conduct - Contracts, Monitoring, User Journeys

Contract Testing

Capture service communication in a contract

Pact terminology:

  • Consumer of a service
  • Provider
  • Pactfiles
  • Broker

Consumer

  1. 🧪 Write unit tests demonstrating usage
  2. 📜 Run tests against a mock to generate contracts
  3. 💾 Upload to Broker during CI

A Pactfile viewed in the Pact Broker UI

Provider

Does this build satisfy the consumer's contracts?

During CI...

  1. 📜 Retrieve mainline contract for each consumer
  2. 🧪 Start service (mock upstream dependencies)
  3. 🚀 Launch Pact mock client
  4. ✔️ Verify service's responses against contract
  5. 💾 Upload verification results to Broker

When a new consumer contract is published...

  1. ⚡ Special CI provider job triggered by Broker
  2. ✔️ Runs provider verification against new contract
  3. 💾 Upload verification result to Broker

From the UK Govt’s (Public) Pact Broker Instance

Time to Deploy!

Pre-

              
pact-broker can-i-deploy
  --pacticipant=pact-consumer
  --version=$CI_COMMIT_SHORT_SHA
  --to-environment=production
              
            

Post-

              
pact-broker record-deployment
  --pacticipant=pact-consumer
  --version=$CI_COMMIT_SHORT_SHA
  --environment=production
              
            

Deploy consumer (when provider isn't deployed)

Deploy provider

Deploy consumer

Deploy consumer (with unsupported changes)

  • Provider States
  • Pending Pacts
  • WIP Pacts
  • Bi-directional Contract Testing

Further Reading