API Study Material

🔹 What is an API?

 An API (Application Programming Interface) allows different parts of a program

or different programs to communicate.
 Two main types:
o Internal APIs – Used within the same application (easier to refactor).
o External APIs – Used by other applications or systems (harder to change
without breaking clients).

🔹 Evolution of APIs

1. Shared Libraries (Early Days):

o Functions/classes linked at compile time.
o Used directly in the same environment.
2. Network APIs (Modern Era):
o APIs available over a network, often used across different machines or
systems.

🔹 RPC & SOAP

 RPC (Remote Procedure Call): Call a function via a URL with parameters.
 SOAP (Simple Object Access Protocol):
o XML-based.
o Requires strict schemas and special tools.
o Works with various protocols (not just HTTP).
o Adds complexity but supports strong validation.

🔹 REST (Representational State Transfer)

 Introduced by Roy Fielding as a simpler alternative to SOAP.

 Treats everything as a resource accessible by a unique URL.
 Uses standard HTTP methods:
o GET, POST, PUT, DELETE, etc.
 Benefits:
o Works naturally with web tools (e.g., browser caching, security).
o Stateless, layered, uniform interface, and client-server separation.
o Easy to use and debug with standard tools.

🔹 SOAP vs REST

Feature SOAP REST

Format XML-based JSON (mostly)
Complexity High Low
Tooling Required Yes (special libs/tools) No (uses standard HTTP tools)
Flexibility Rigid, schema-driven Flexible, lightweight
Usage Enterprise, complex systems Web apps, mobile apps
API Responsibilities
🔸 Avoid One-Size-Fits-All Design

 A single API serving both browser apps and server-to-server processes becomes
overly complex.
 Complexity leads to bugs and harder maintenance.

🔸 Use Layered API Architecture

1. Backend-for-Frontend (BFF) Pattern:

 Create a specific API layer just for the front end (e.g., browser apps).
 This layer:
o Talks to the main backend API.
o Acts as a translator to simplify frontend interaction.
o Adds validation and authentication close to the user.

2. Core Backend API:

 Locked down.
 Used for server-to-server communication.
 Simplified and trusted due to extra security at the edge (BFF layer).

✅ Benefits of Layered API Design

 Simplified and focused APIs.

 Easier testing and debugging.
 Clear separation of concerns.
 Stronger security through isolation.
 Better performance via caching and optimization at each layer.

Designing a REST API – Summary

1. API Design Philosophy:
o Similar to GUI design: focus on user experience and predictability.
o Focus on clarity, structure, and consistency of data.
o Ideal design is sometimes limited by backend performance.
2. URL Structure & HATEOAS:
o RESTful APIs may use HATEOAS (Hypermedia as the Engine of Application
State), guiding users through links in responses.
o Though conceptually powerful, HATEOAS is often skipped due to
performance concerns, developers often rely on known URL patterns instead.
o Including full URIs (rather than just IDs) improves flexibility and reduces
assumptions.
3. Content Negotiation & Best Practices:
o REST APIs can support multiple formats (e.g., JSON, XML) using content
negotiation (Accept header).
o However, this may complicate caching and implementation.
o For simplicity, many APIs serve only JSON and separate frontend/backend
domains.
o Following the KISS principle—serving only JSON and using distinct domains
for API vs frontend—simplifies development.
 o Custom MIME types (like application/vnd.example+json) can be difficult
for clients to handle and often aren’t necessary.
4. Serialization Formats & Security:
o XML historically has a strong schema ecosystem for validation.
o XML has long held an advantage in API design due to well-supported
schema definitions (e.g., XSD), allowing robust validation and structure
enforcement.
o JSON, while simpler, lacks a universal schema system
o JSON Schema is increasingly popular and works well with tools like
Swagger and RAML for API documentation and validation
o Some frameworks support multiple serialization formats (XML, JSON,
YAML, etc.) and can convert between them automatically.
o While flexible, this adds complexity and increases the surface area for
security vulnerabilities / security risk
 YAML, while powerful, can execute code when improperly
deserialized—leading to major breaches (e.g., GitHub, Equifax).
 Even XML has known exploits like the "Billion Laughs" attack, which
can crash parsers.
5. Best practices:
o Stick to one format (usually JSON) to reduce complexity.
o Enable parser safe modes.
o Treat all input as untrusted.
o Understand your deserialization library's behavior.
6. Implement robust error handling and input validation.Statelessness &
Idempotence:
o REST APIs should be stateless—each request stands alone, with no client
context stored on the server.
o This supports scalability and simplifies architecture.
o Idempotence: repeating the same request should yield the same result (e.g.,
GET, PUT, DELETE are idempotent; POST, PATCH are not).
o Try to maintain symmetry: if you GET a resource, the response body should
be usable in a PUT request without modification.
o When data includes linked resources, use flags like ?expand=true to
include inline data and reduce client-side requests.
o But note: the expanded version may not be suitable for PUT as-is.
7. HTTP Methods Overview:
o GET: retrieve data (cacheable, idempotent).
o PUT: replace a resource entirely (can be used to create or update).
o POST: create or trigger actions (not idempotent).
o PATCH: update parts of a resource (helps avoid PUT's full-replacement
approach).
o DELETE: remove a resource.
o HEAD/OPTIONS: support metadata and preflight checks (e.g., CORS).
8. Concurrency Handling:
o To avoid conflicts (e.g., during PUT), use conditional headers like:
 ETag with If-Match
 Last-Modified with If-Not-Modified-Since
9. HTTP Request & Response:
o Requests and responses include headers and an optional body.
o Headers carry metadata like content type, language, referer, etc.
o
10. HTTP Status Codes (Important Categories)

 2xx (Success):
o 200 OK: Successful standard response.
o 201 Created: Resource created; often with Location header.
 3xx (Redirection):
o 301/308: Permanent redirects.
 o 302/307/303: Temporary or method-changing redirects.
 4xx (Client Errors):
o 400 Bad Request: Invalid syntax or validation issues.
o 401 Unauthorized / 403 Forbidden: Access control errors.
o 404 Not Found / 410 Gone: Missing or deleted resources.
 5xx (Server Errors): Indicate problems on the server side.

APIs often use structured JSON for error messages, and standards like RFC 7807 exist for
consistent formatting.

o Redirection behavior (especially 3xx) can vary due to browser quirks.

11. Request/Response Structure & Expansion:
o Maintain parity: a GET response should ideally be PUT-able without
changes.
o For performance, use query flags like ?expand=true to inline linked resource
data, reducing extra GETs.
o Expanded resources may not be PUT-able as-is—design accordingly.
12. Schema Evolution & Compatibility:

 Design with change in mind:

o Adding new fields should not break existing clients.
o Clients should ignore unknown fields.
o Major schema changes might require supporting both old and new formats
temporarily.
 Avoid JSON lists as top-level resources. Use objects instead to allow for future
additions (e.g., pagination metadata).

11. Versioning Strategies:

 Include versioning from the start to anticipate changes.

 Common methods:
o URL-based: https://api.example.com/v1/products
o MIME type versioning: application/vnd.example.v1+json
o Some advocate skipping versioning in favor of entirely new endpoints or
services.
 Decommission old versions only after all clients have migrated.

Securing Your API – Key Concepts and Approaches

1. Authentication vs. Authorization
 Authentication (AuthN): Verifying who someone is.
 Authorization (AuthZ): Verifying what they’re allowed to do.

2. Basic Access Control

 Firewalling: Restricting API access to specific servers or IP ranges.
o Limitation: If one server is compromised, full access may be gained.
 Basic Auth (via HTTP headers): Username/password-based access control.
o Easily configured in servers like Apache or nginx.

3. TLS Client Certificates

 Mutual TLS: Clients present a certificate for identity verification.

 Pros: Hard to brute-force, centrally revocable, and OS-integrable.
 Cons: Requires complex PKI setup (X.509).
4. API Keys
 Passed via headers or query parameters.
 Pros: Simple to implement.
 Cons: Susceptible to leakage (e.g., decompiling mobile apps).

5. Whitelisting & Configuration

 Grant access only to users explicitly listed.
 Can be embedded in app or config.
o Issue: Requires redeployments for changes.
o Visibility: Hard to audit access across systems.

6. Central Authorization Service

 External service checks access rights per request.

 Good for large systems needing dynamic, centralized control.

7. API Gateways
 Sits in front of APIs, centralizing authn/authz.
 Can enforce:
o API Key validation
o URL-based access control
o Rate limiting
 Risks: Must ensure APIs aren’t accessible if gateway is bypassed.

8. OAuth & JWT

 OAuth (esp. OAuth 2.0):
o Used in user-delegated access scenarios.
o Tokens have scoped permissions.
o Suitable for client-facing applications.
 JWT (JSON Web Token):
o A signed token carrying claims like identity and permissions.
o Can be verified offline using the issuer’s public key.

9. Rate Limiting

 Prevents abuse and ensures fair usage.

 Limits requests per user/IP over time window.
 Often built into API gateways (e.g., 1000 requests/hour).

Best Practices

 Combine methods: Use TLS, JWT/OAuth, and API keys where appropriate.
 Use a gateway for centralized security and visibility.
 Rotate credentials regularly.
 Audit access and monitor for anomalies.
 Rate-limit sensitive or high-cost endpoints.
 Avoid embedding sensitive credentials in client apps.
Event-Based APIs.
1. Traditional RESTful APIs vs. Event-Based APIs
 RESTful APIs: Focus on requesting and changing the state of an external system.
 Event-Based APIs: Designed for notifying when a change occurs in the system,
allowing clients to react to state changes as they happen without the need to
constantly poll for updates.

2. Use Cases for Event-Based APIs

 Backend Applications:
o For example, a shipping label printing system that gets triggered to print a
label when an order is picked.
o Helps distribute responsibility across different components when changes
need to be communicated to multiple systems.
 Frontend Applications:
o Example: A web chat application that updates users in real-time when
someone sends a new message in a chat room.
o Instead of polling frequently, the server pushes updates to the client through
an open connection.

3. Challenges with Polling (RESTful Model)

 Polling involves making regular API requests to check if the state has changed,
which can lead to:
o Inefficiency: A lot of unnecessary requests for infrequent changes.
o Delay: New data might not be visible until the next polling interval, creating a
poor user experience.

4. Event-Driven Solutions
 Comet (2000s):
o An early workaround where the server slowly sends page content, essentially
mimicking a long-running connection using iframes.
 WebSockets (2011):
o An HTML5 feature that enables two-way communication between a client
and server, removing the need for polling or Comet.
o The server can push updates directly to the client whenever the state
changes, making it more efficient and real-time.

5. Message Queues for Back-End Systems

 For backend systems, message queue technologies (e.g., RabbitMQ, Kafka) can be
used for event-driven communication.
 Event Queue: The system can process events as they arrive, allowing for
decoupled, reactive system behavior.
6. Event Replay and State Management
 Replaying Events:
o Instead of depending on the initial state from upstream systems, applications
can replay all past events to rebuild the current state, allowing for:
 Fault tolerance (e.g., rebuilding after data corruption).
 Scalability: For large distributed systems, events can be reprocessed
as needed.
 Event Pruning:
o Over time, some events become irrelevant (e.g., the final address update
supersedes older address updates).
o Only the most recent or relevant events are retained, reducing the storage
overhead.

7. Advantages & Challenges of Event-Based APIs

 Advantages:
o Real-time Updates: Efficiently pushing changes to clients as they occur.
o Resilience: Event-driven systems are less reliant on a single point of failure,
as the event broker or queue can handle events independently.
 Challenges:
o Single Point of Failure: While distributed systems can eliminate one API
failure point, the event broker itself becomes a critical component.
o Complexity: Managing event histories and handling events that are
superseded or outdated requires sophisticated event processing.

8. Best Practices

 Event Brokers: Use reliable message brokers (e.g., Kafka, RabbitMQ) to ensure
event delivery and reliability.
 Event Versioning: Maintain backward compatibility and track event versions,
especially when evolving the event schema.
 Event Deduplication: Handle potential duplicate events (e.g., re-processing the
same event after a failure) to avoid unintended actions (like double payments).

Discovering APIs
 Hardcoding vs. Configuration:

 For a simple app, hardcoding the API URL may work initially, but it’s not scalable. A
more flexible approach is to configure the API URL through a configuration file or
environment variables.
 This allows the API to be deployed in different environments (development, staging,
production), with each environment possibly using a different URL.

 DNS and Load Balancers:

  Assigning sensible DNS names to the API, like productapi.dev.example.com for
development and productapi.prod.example.com for production, makes the setup
more organized.
 If there are multiple servers, the URL could point to a load balancer that routes
requests to one of the available servers, which helps manage traffic and provides
scalability.

 Limitations of Load Balancers:

 The approach of using DNS names and load balancers works for general use cases,
but it doesn’t work well if you need to reference a specific instance of a resource. In
such cases, DNS-based abstraction or load balancers might not suffice.

 Service Discovery:

 When environments are frequently provisioned or updated, managing API URLs

becomes more complex. Using a service discovery tool can solve this problem by
dynamically discovering the URLs of services.
 These tools allow components to register themselves and be queried by others,
making the process of locating services more flexible. This reduces the need to
hardcode URLs or manually configure them every time a new environment is created.

 Caution with Service Discovery:

 Service discovery tools can become a critical dependency, so they should be used
only when absolutely necessary. You need to be confident that the benefits they
provide—like reducing the complexity of managing multiple environments—are
worth the additional layer of infrastructure.

Using APIs:
 API Interaction Basics: Once an API is deployed and accessible, the client needs to
interact with it by making HTTP requests, typically with credentials, and parsing the
responses. However, interacting with an API requires considering potential failures and
handling errors effectively.

 Error Handling: APIs may produce both expected errors (e.g., 404 Not Found) and
unexpected errors (e.g., 500 Internal Server Error). It's important to manage these errors in a
way that provides clear feedback to the caller, often using modules that centralize API
interactions and error handling.

 Complex Error Handling: If an API call is part of a sequence (e.g., payment API
followed by a shipping system), failure in one step can affect the entire system. Handling
such errors can include undoing actions, notifying users or operations teams, or retrying the
request. A retry mechanism can also be effective when appropriate.

 API Monitoring: It's crucial to monitor interactions with external systems. This includes
logging errors, tracking response times, and counting successful/failed requests. Monitoring
helps in debugging and setting thresholds for system performance.

 Rate Limiting and Circuit Breakers: If an API has rate limits, it’s useful to manage
requests by integrating a circuit breaker or queuing requests to retry after the limit resets.
 Caching: Caching improves performance and helps with rate limits. However, improper
caching can lead to stale information or errors. A simple approach is to store successful
responses in a cache, and in case of errors (like 404), you might cache those as well to avoid
repeated checks.

 Negative Caching: This involves caching error responses (like 404) to prevent repeated
hits on non-existent resources. While it can be effective in certain cases, negative caching can
cause issues, such as caching an incorrect error state.

 Thundering Herd Problem: This occurs when multiple requests are made while a cache
is expired, overwhelming the backend. The solution is to mark items as stale and allow the
first client to refresh the cache, while others get the stale data. This is known as "stale-while-
revalidate."

 Stale-on-Error: When the API is unavailable or crashes, stale data in the cache can be
used to maintain user experience. However, if the timeliness of information is critical, it
might be better to show an error rather than outdated information.