diff --git a/README.md b/README.md index 80c53ae..c142d93 100644 --- a/README.md +++ b/README.md @@ -56,6 +56,11 @@ foss42 APIs project has 3 parts: - [foss42/api](https://github.com/foss42/api): The FastAPI app which serves the APIs. - [foss42/foss42-core](https://github.com/foss42/foss42-core): The open source core python library which has the algorithms, the data and does all the heavy-lifting. +## Repository structure + +- `src/` — the hosted Open Source APIs (FastAPI app), with `tests/` as their test suite. +- `grpc/` and `mqtt/` — self-contained, **local protocol test rigs** for API Dash's gRPC and MQTT clients (they are not hosted). Each has its own `README.md`, `tests/` and `requirements-dev.txt`, and runs via Docker. + ## How to Run Locally 1. **Clone the repository:** diff --git a/docs/grpc/README.md b/docs/grpc/README.md index 5787bad..3f65200 100644 --- a/docs/grpc/README.md +++ b/docs/grpc/README.md @@ -67,7 +67,7 @@ Azure/production infra is wired up separately. `grpc/docker-compose.yml` -- the image (generates the Protobuf stubs at build time and mints a self-signed TLS cert on first start) and the one-command rig. - `docs/grpc/` -- this README plus per-scenario pages (reflection, unary, the - three streaming modes, metadata, errors, TLS). + three streaming modes, metadata, auth, errors, TLS). ## Run it @@ -86,6 +86,30 @@ This starts one service: Stop it with `Ctrl-C`, or run detached with `-d` and stop via `docker compose -f grpc/docker-compose.yml down`. +## Tests + +A pytest round-trip suite (`tests/grpc/test_grpc.py`) exercises every method of +`apidash.test.TestService` against a running server -- the gRPC analogue of the +MQTT rig's `tests/mqtt/test_mqtt.py`. It covers reflection, unary (`Echo`, +`GetRandomUser`), all three streaming modes (`StreamTicks`, `SumNumbers`, +`Chat`), request/response metadata (`EchoMetadata`), auth-via-metadata +(`SecureEcho`) and error status codes (`RaiseError`). + +The Protobuf stubs are generated on the fly at test-collection time from +`grpc/proto/apidash_test.proto` (via `grpc_tools.protoc`), so nothing generated +is committed. Run it: + +``` +docker compose -f grpc/docker-compose.yml up --build -d # start the server +pip install -r requirements-dev.txt # grpcio + tools + reflection +pytest tests/grpc/test_grpc.py +``` + +The whole module **skips gracefully** (it does not fail) when grpcio / +grpcio-tools are missing or when no server is reachable at `localhost:9000`, so +CI without a server stays green. Point it elsewhere with the `GRPC_HOST` / +`GRPC_PORT` environment variables. + ## Endpoints | Transport | Address | TLS | Notes | @@ -131,7 +155,8 @@ that exercises it and the mock/random data it returns: | Server streaming | `apidash.test.TestService/StreamTicks` | A stream of `Tick`s with random values | [server_streaming](server_streaming.md) | | Client streaming | `apidash.test.TestService/SumNumbers` | `sum` / `count` / `average` of the numbers you send | [client_streaming](client_streaming.md) | | Bidirectional streaming | `apidash.test.TestService/Chat` | Each message echoed back, server-timestamped | [bidi](bidi.md) | -| Metadata / auth-via-metadata | `apidash.test.TestService/EchoMetadata` | The request metadata, echoed back | [metadata](metadata.md) | +| Metadata / auth-via-metadata | `apidash.test.TestService/EchoMetadata` | The request metadata echoed back, plus response initial + trailing metadata | [metadata](metadata.md) | +| Auth (Bearer / API key) | `apidash.test.TestService/SecureEcho` | Echoes your message only with valid credentials; else `UNAUTHENTICATED` | [auth](auth.md) | | Errors / status codes | `apidash.test.TestService/RaiseError` | Fails with the gRPC status code you request | [errors](errors.md) | | TLS transport | any method on `localhost:9001` | Same methods over TLS | [tls](tls.md) | @@ -139,7 +164,7 @@ Full method list: | Service | Methods | | ----------- | ----------- | -| `apidash.test.TestService` | `Echo`, `GetRandomUser`, `StreamTicks`, `SumNumbers`, `Chat`, `EchoMetadata`, `RaiseError` | +| `apidash.test.TestService` | `Echo`, `GetRandomUser`, `StreamTicks`, `SumNumbers`, `Chat`, `EchoMetadata`, `SecureEcho`, `RaiseError` | | `grpc.reflection.v1alpha.ServerReflection` | `ServerReflectionInfo` (used by **Reflect**) | > **Reflection version note:** the server registers the standard **v1alpha** @@ -155,7 +180,8 @@ Full method list: - [server_streaming](server_streaming.md) -- `StreamTicks` (one request -> stream) - [client_streaming](client_streaming.md) -- `SumNumbers` (stream -> one response) - [bidi](bidi.md) -- `Chat` (bidirectional streaming echo) -- [metadata](metadata.md) -- `EchoMetadata` (custom headers + auth-via-metadata) +- [metadata](metadata.md) -- `EchoMetadata` (custom request headers + response metadata) +- [auth](auth.md) -- `SecureEcho` (auth-protected: Bearer token / API key) - [errors](errors.md) -- `RaiseError` (choose the gRPC status code) - [tls](tls.md) -- gRPC over TLS on `9001` diff --git a/docs/grpc/auth.md b/docs/grpc/auth.md new file mode 100644 index 0000000..eb0e686 --- /dev/null +++ b/docs/grpc/auth.md @@ -0,0 +1,109 @@ +--- +protocol: grpc +title: gRPC Auth (SecureEcho) +desc: SecureEcho is an auth-protected unary call -- it accepts the request only when you send a valid Bearer token or API key as metadata, and fails UNAUTHENTICATED otherwise, so you can test API Dash's Auth tab and API-key auth. +path: grpc/auth +--- + +gRPC has no separate "auth" mechanism -- credentials travel as +[metadata](metadata.md) (the gRPC equivalent of HTTP headers). A server enforces +auth by reading that metadata and rejecting the call when it's missing or wrong. + +`apidash.test.TestService/SecureEcho` is built for testing exactly this. It +behaves like [`Echo`](unary.md) -- echoes your `message` back with a server time +and sequence number -- but **only if the call carries valid credentials**. +Otherwise it fails with gRPC status **`16 UNAUTHENTICATED`**. + +## What it tests + +That API Dash can attach credentials to a gRPC call -- a **Bearer token** via +the Auth tab (which becomes an `authorization` metadata entry) or an **API key** +as a custom metadata header -- and that it surfaces the `UNAUTHENTICATED` status +when they're absent or wrong. + +## Method + +``` +apidash.test.TestService/SecureEcho # EchoRequest{message} -> EchoResponse{message, server_time, seq} +``` + +The request/response shapes are identical to `Echo`; the difference is the +credential check on the incoming metadata. On success the echoed `message` is +prefixed with `[authenticated]` so you can tell the two apart. + +## Accepted credentials + +The call is accepted if **EITHER** of these is present in the request metadata: + +| Metadata key | Value | How to send it in API Dash | +| ----------- | ----------- | ----------- | +| `authorization` | `Bearer test-token` | Auth tab -> Bearer token = `test-token` | +| `x-api-key` | `test-apikey` | Add a metadata / header row `x-api-key` = `test-apikey` | + +Anything else -- no credentials, a wrong token, a wrong key -- is rejected. + +## Expected behavior + +| You send | Result | +| ----------- | ----------- | +| Metadata `authorization: Bearer test-token` | Success -- `message` echoed as `[authenticated] ` | +| Metadata `x-api-key: test-apikey` | Success -- same as above | +| No credentials | Call fails with status `16 UNAUTHENTICATED` | +| Wrong token / key (e.g. `Bearer nope`) | Call fails with status `16 UNAUTHENTICATED` | + +The failure message is: +`missing or invalid credentials -- send 'authorization: Bearer test-token' or 'x-api-key: test-apikey'`. + +## Test it in API Dash + +1. **Reflect** against `localhost:9000` and pick + `apidash.test.TestService/SecureEcho`. +2. Set the request to `{"message": "hello"}` and **Send** with no credentials. + - **Expected:** the call fails and API Dash shows status + **`16 UNAUTHENTICATED`** with the missing-credentials message. +3. Open the **Auth** tab, choose **Bearer** and enter the token `test-token` + (API Dash puts `authorization: Bearer test-token` on the wire). **Send**. + - **Expected:** success -- the response echoes + `[authenticated] hello` with a `server_time` and `seq`. +4. Alternatively, instead of the Auth tab, add a metadata row `x-api-key` = + `test-apikey` and **Send**. + - **Expected:** the same success response. + +## Sample Usage + +### grpcurl + +Pass credentials as metadata with `-H "key: value"`: + +``` +grpcurl -plaintext -d '{"message": "hi"}' \ + -H "authorization: Bearer test-token" \ + localhost:9000 apidash.test.TestService/SecureEcho +# { +# "message": "[authenticated] hi", +# "server_time": "2026-08-24T...Z", +# "seq": 1 +# } +``` + +With an API key instead: + +``` +grpcurl -plaintext -d '{"message": "hi"}' \ + -H "x-api-key: test-apikey" \ + localhost:9000 apidash.test.TestService/SecureEcho +# { "message": "[authenticated] hi", ... } +``` + +With no credentials it fails: + +``` +grpcurl -plaintext -d '{"message": "hi"}' \ + localhost:9000 apidash.test.TestService/SecureEcho +# ERROR: +# Code: Unauthenticated +# Message: missing or invalid credentials -- send 'authorization: Bearer test-token' or 'x-api-key: test-apikey' +``` + +For TLS use `localhost:9001` -- see [tls](tls.md). To just inspect which metadata +reaches the server (without the auth check), use [`EchoMetadata`](metadata.md). diff --git a/docs/grpc/metadata.md b/docs/grpc/metadata.md index d51ce8b..ab8e703 100644 --- a/docs/grpc/metadata.md +++ b/docs/grpc/metadata.md @@ -1,7 +1,7 @@ --- protocol: grpc title: gRPC Metadata & Auth -desc: EchoMetadata reflects the request metadata (gRPC headers) back to you, so you can verify custom headers and auth-via-metadata (e.g. an authorization token) are sent. +desc: EchoMetadata reflects the request metadata (gRPC headers) back to you AND sends response metadata (initial + trailing) back, so you can verify both custom request headers / auth-via-metadata and the response-metadata view. path: grpc/metadata --- @@ -10,15 +10,26 @@ alongside every call. It's how clients send things like an `authorization` token, a request id, or an API key -- gRPC has no separate "auth" mechanism, so **auth is done via metadata**. -`apidash.test.TestService/EchoMetadata` is built for testing this: it **returns -the request metadata back to you** in the response, so you can confirm exactly -which headers API Dash put on the wire. +`apidash.test.TestService/EchoMetadata` is built for testing this in **both +directions**: + +- **Request -> server:** it **returns the request metadata back to you** in the + response body, so you can confirm exactly which headers API Dash put on the + wire. +- **Server -> response:** it also **sends response metadata back** -- initial + metadata (`x-server`, `x-echoed-count`) and trailing metadata (`x-trailer`) -- + so API Dash's **response metadata / headers view** is testable too. ## What it tests That API Dash attaches custom metadata to a gRPC call (including an `authorization` header for auth-via-metadata) and that the values arrive at the -server intact. +server intact -- **and** that it surfaces the metadata the server sends back on +the response (initial + trailing). + +> For an auth check that actually **rejects** calls without valid credentials +> (rather than just echoing whatever you send), see [auth](auth.md) +> (`SecureEcho`). ## Method @@ -41,6 +52,23 @@ This makes it a quick way to prove an auth token or any custom header is actuall being sent -- if it comes back, the server received it. (Note gRPC lowercases metadata keys on the wire, so `Authorization` comes back as `authorization`.) +## Response metadata (server -> client) + +Besides echoing the request metadata into the response **body**, the server +attaches metadata to the **response** itself -- the mirror image of what you +send. These are surfaced in API Dash's response metadata / headers view, not in +the message body: + +| Metadata | When it arrives | Value | +| ----------- | ----------- | ----------- | +| `x-server` | initial metadata (response headers) | `apidash-grpc-test` | +| `x-echoed-count` | initial metadata (response headers) | number of request metadata pairs the server received | +| `x-trailer` | trailing metadata (response trailers) | `ok` | + +So a single `EchoMetadata` call exercises metadata in both directions: +request -> server (echoed in the body) **and** server -> response (the initial + +trailing metadata above). + ## Test it in API Dash 1. **Reflect** against `localhost:9000` and pick @@ -48,10 +76,13 @@ metadata keys on the wire, so `Authorization` comes back as `authorization`.) 2. Add metadata rows to the request, e.g. `authorization` / `Bearer test-token` and `x-request-id` / `abc123`. Leave the request body empty. 3. **Send**. - - **Expected:** the response `metadata` map contains + - **Expected (response body):** the response `metadata` map contains `authorization: Bearer test-token` and `x-request-id: abc123` (plus transport headers like `user-agent`). This is the pattern for testing **auth-via-metadata** against any real gRPC service. + - **Expected (response metadata / headers view):** `x-server: + apidash-grpc-test` and `x-echoed-count: ` in the initial metadata, and + `x-trailer: ok` in the trailing metadata. ## Sample Usage @@ -74,6 +105,23 @@ grpcurl -plaintext \ # } ``` +Add `-v` to also see the **response** metadata grpcurl received -- the server's +initial and trailing metadata: + +``` +grpcurl -plaintext -v \ + -H "authorization: Bearer test-token" \ + localhost:9000 apidash.test.TestService/EchoMetadata +# Response headers received: +# x-server: apidash-grpc-test +# x-echoed-count: 2 +# ... +# Response trailers received: +# x-trailer: ok +``` + In API Dash: add metadata rows to the request, call `EchoMetadata`, and confirm -the same key/value pairs appear in the response. For TLS use `localhost:9001` -- -see [tls](tls.md). +the same key/value pairs appear in the response body -- and that `x-server` / +`x-echoed-count` / `x-trailer` appear in the response metadata view. For TLS use +`localhost:9001` -- see [tls](tls.md). For an auth check that rejects invalid +credentials, see [auth](auth.md). diff --git a/grpc/README.md b/grpc/README.md new file mode 100644 index 0000000..e802aea --- /dev/null +++ b/grpc/README.md @@ -0,0 +1,26 @@ +# gRPC test rig + +A self-contained, local gRPC test server (`apidash.test.TestService`) for +exercising API Dash's gRPC client -- reflection, unary, all three streaming +modes, metadata, auth-via-metadata and error codes. It is **not** part of the +hosted Open Source APIs; it only runs locally via Docker. + +## Setup & run the server + +```bash +docker compose -f grpc/docker-compose.yml up --build +``` + +Serves on `localhost:9000` (plaintext) and `localhost:9001` (TLS, self-signed). + +## Run the tests + +```bash +pip install -r grpc/requirements-dev.txt +pytest grpc/tests +``` + +The suite generates the Protobuf stubs on the fly and **skips gracefully** when +grpcio/-tools are missing or no server is reachable on `localhost:9000`. + +See [`docs/grpc/`](../docs/grpc/) for the per-feature pages. diff --git a/grpc/proto/apidash_test.proto b/grpc/proto/apidash_test.proto index 0830475..4062b10 100644 --- a/grpc/proto/apidash_test.proto +++ b/grpc/proto/apidash_test.proto @@ -14,7 +14,8 @@ // StreamTicks server stream N random "ticks" (a price/sensor-style feed) // SumNumbers client stream sum / count / average of the numbers you send // Chat bidi stream echoes each message back, stamped server-side -// EchoMetadata unary reflects the call's gRPC metadata back to you +// EchoMetadata unary reflects request metadata + sends response metadata +// SecureEcho unary like Echo, but requires auth-via-metadata credentials // RaiseError unary fails with the gRPC status code you request // // The server (server.py) has server reflection enabled, so API Dash's Reflect @@ -45,10 +46,17 @@ service TestService { // echoed back with a server-applied timestamp. rpc Chat (stream ChatMessage) returns (stream ChatMessage); - // Unary: reflects the call's gRPC metadata (headers) back to you, for - // testing custom metadata and auth-via-metadata (e.g. an authorization token). + // Unary: reflects the call's gRPC metadata (headers) back to you, AND sends + // response metadata (initial + trailing) back, for testing custom metadata, + // auth-via-metadata (e.g. an authorization token) and response-header views. rpc EchoMetadata (Empty) returns (MetadataResponse); + // Unary: like Echo, but AUTH-PROTECTED via metadata. Accepts the call only if + // metadata `authorization` == "Bearer test-token" OR `x-api-key` == + // "test-apikey"; otherwise fails UNAUTHENTICATED. Exercises the Auth tab + // (Bearer token -> authorization metadata) and API-key auth. + rpc SecureEcho (EchoRequest) returns (EchoResponse); + // Unary: deliberately fails with the gRPC status code you request, for // testing how the client surfaces non-OK statuses. code = a gRPC status code. rpc RaiseError (ErrorRequest) returns (Empty); diff --git a/grpc/requirements-dev.txt b/grpc/requirements-dev.txt new file mode 100644 index 0000000..ec6a0f1 --- /dev/null +++ b/grpc/requirements-dev.txt @@ -0,0 +1,3 @@ +grpcio # grpc/tests: local gRPC server round-trip tests (not needed by the prod app) +grpcio-tools # grpc/tests: generates the Protobuf stubs on the fly from grpc/proto (not needed by the prod app) +grpcio-reflection # grpc/tests: client-side reflection listing in the round-trip tests (not needed by the prod app) diff --git a/grpc/server.py b/grpc/server.py index 1f9180c..235339e 100644 --- a/grpc/server.py +++ b/grpc/server.py @@ -15,7 +15,8 @@ StreamTicks server stream N random "ticks" (a price/sensor-style feed) SumNumbers client stream sum / count / average of the numbers you send Chat bidi stream echoes each message back, stamped server-side - EchoMetadata unary reflects the call's gRPC metadata back to you + EchoMetadata unary reflects request metadata + sends response metadata + SecureEcho unary like Echo, but requires auth-via-metadata credentials RaiseError unary fails with the gRPC status code you request Two listeners are opened (same service on both, only the transport differs): @@ -88,6 +89,12 @@ ] EMAIL_DOMAINS = ["example.com", "test.dev", "mail.invalid", "apidash.dev"] +# --- Accepted credentials for SecureEcho (auth-via-metadata) ----------------- +# The call is accepted if EITHER of these matches. Fixed test values -- this is +# a fixture, not a real credential store. (See docs/grpc/auth.md.) +VALID_BEARER = "Bearer test-token" # metadata "authorization" +VALID_API_KEY = "test-apikey" # metadata "x-api-key" + def log(msg): """Timestamped stdout logging (see `docker compose logs server`).""" @@ -193,11 +200,11 @@ def Chat(self, request_iterator, context): ts=now_iso(), # server stamps the reply ) - # --- Unary: echo the request's gRPC metadata ---------------------------- + # --- Unary: echo request metadata + send response metadata -------------- def EchoMetadata(self, request, context): # invocation_metadata() is the gRPC equivalent of request headers. This # is how auth-via-metadata (e.g. an `authorization` token) is tested: - # whatever the client sent comes straight back. + # whatever the client sent comes straight back in the response body. out = {} for key, value in context.invocation_metadata(): # Binary metadata keys end in "-bin" and carry bytes; render those @@ -205,9 +212,45 @@ def EchoMetadata(self, request, context): if isinstance(value, bytes): value = value.decode("utf-8", "replace") out[key] = value + + # Also send metadata BACK (server -> client) so the response-metadata / + # headers view is testable: initial metadata is delivered with the + # response headers, trailing metadata with the final trailers. + context.send_initial_metadata( + [("x-server", "apidash-grpc-test"), ("x-echoed-count", str(len(out)))] + ) + context.set_trailing_metadata([("x-trailer", "ok")]) + log(f"EchoMetadata: {len(out)} metadata entrie(s)") return pb.MetadataResponse(metadata=out) + # --- Unary: auth-protected echo (auth-via-metadata) --------------------- + def SecureEcho(self, request, context): + # gRPC has no built-in auth; credentials travel as metadata. Accept the + # call if EITHER a Bearer token OR an API key matches; otherwise abort + # UNAUTHENTICATED. This exercises API Dash's Auth tab (Bearer token -> + # `authorization` metadata) and API-key auth. + md = dict(context.invocation_metadata()) + authorized = ( + md.get("authorization") == VALID_BEARER + or md.get("x-api-key") == VALID_API_KEY + ) + if not authorized: + log("SecureEcho: DENIED (missing or invalid credentials)") + context.abort( + grpc.StatusCode.UNAUTHENTICATED, + "missing or invalid credentials -- send 'authorization: Bearer " + "test-token' or 'x-api-key: test-apikey'", + ) + + seq = self._next_seq() + log(f"SecureEcho(seq={seq}) message={request.message!r} [authenticated]") + return pb.EchoResponse( + message=f"[authenticated] {request.message}", + server_time=now_iso(), + seq=seq, + ) + # --- Unary: fail with the requested gRPC status code -------------------- def RaiseError(self, request, context): # Map the requested numeric code to a grpc.StatusCode. Anything out of diff --git a/grpc/tests/test_grpc.py b/grpc/tests/test_grpc.py new file mode 100644 index 0000000..14e800d --- /dev/null +++ b/grpc/tests/test_grpc.py @@ -0,0 +1,282 @@ +""" +Integration tests for the local gRPC test rig (grpc/docker-compose.yml). + +These talk to a REAL gRPC server over HTTP/2 (grpcio against localhost:9000), +which is the only way to genuinely exercise reflection, the three streaming +modes, metadata / auth-via-metadata and gRPC status codes -- the same reason we +ship a Dockerized `apidash.test.TestService` (grpc/server.py) rather than a +hand-rolled FastAPI route. + +The Protobuf stubs are generated ON THE FLY at import time from +`grpc/proto/apidash_test.proto` (via grpc_tools.protoc) into a temp dir that is +put on sys.path -- so nothing generated is committed, exactly like the Docker +image builds them at build time. + +The whole module SKIPS gracefully when: + - grpcio is not installed, or + - grpcio-tools is not installed (can't generate the stubs), or + - no server is reachable at GRPC_HOST:GRPC_PORT (default localhost:9000), +so CI without a server still passes. To run them for real: + + docker compose -f grpc/docker-compose.yml up --build -d + pip install -r grpc/requirements-dev.txt + pytest grpc/tests + +Each test is independent and hits one feature of the service. +""" + +import os +import socket +import sys +import tempfile + +import pytest + +# Skip cleanly if grpcio / grpcio-tools are not installed. +grpc = pytest.importorskip("grpc", reason="grpcio not installed") +protoc = pytest.importorskip( + "grpc_tools.protoc", reason="grpcio-tools not installed (needed to gen stubs)" +) + +GRPC_HOST = os.environ.get("GRPC_HOST", "localhost") +GRPC_PORT = int(os.environ.get("GRPC_PORT", "9000")) +GRPC_TARGET = f"{GRPC_HOST}:{GRPC_PORT}" + + +def _server_reachable(host, port, timeout=1.0): + try: + with socket.create_connection((host, port), timeout=timeout): + return True + except OSError: + return False + + +def _find_repo_root(start): + """Walk up from `start` until we find grpc/proto/apidash_test.proto.""" + d = os.path.abspath(start) + while True: + if os.path.isfile(os.path.join(d, "grpc", "proto", "apidash_test.proto")): + return d + parent = os.path.dirname(d) + if parent == d: # reached the filesystem root + return None + d = parent + + +_REPO_ROOT = _find_repo_root(os.path.dirname(__file__)) +if _REPO_ROOT is None: + pytest.skip( + "Could not locate grpc/proto/apidash_test.proto relative to this test.", + allow_module_level=True, + ) + +# Skip the entire module if there is no server to talk to. This keeps CI green +# when no gRPC server is running. +if not _server_reachable(GRPC_HOST, GRPC_PORT): + pytest.skip( + f"No gRPC server reachable at {GRPC_TARGET} " + "(start it with `docker compose -f grpc/docker-compose.yml up --build`)", + allow_module_level=True, + ) + + +def _generate_stubs(repo_root): + """Run grpc_tools.protoc to generate the pb2 stubs into a temp dir on sys.path.""" + proto_dir = os.path.join(repo_root, "grpc", "proto") + proto_file = os.path.join(proto_dir, "apidash_test.proto") + out_dir = tempfile.mkdtemp(prefix="apidash_grpc_stubs_") + rc = protoc.main( + [ + "grpc_tools.protoc", + f"-I{proto_dir}", + f"--python_out={out_dir}", + f"--grpc_python_out={out_dir}", + proto_file, + ] + ) + if rc != 0: + raise RuntimeError(f"grpc_tools.protoc failed with exit code {rc}") + if out_dir not in sys.path: + sys.path.insert(0, out_dir) + return out_dir + + +_generate_stubs(_REPO_ROOT) + +# These import names come from the generated stubs (put on sys.path above). +import apidash_test_pb2 as pb # noqa: E402 +import apidash_test_pb2_grpc as pb_grpc # noqa: E402 + + +# --------------------------------------------------------------------------- # +# Fixtures +# --------------------------------------------------------------------------- # +@pytest.fixture(scope="module") +def channel(): + """A plaintext gRPC channel to the test server, ready before any test runs.""" + ch = grpc.insecure_channel(GRPC_TARGET) + try: + grpc.channel_ready_future(ch).result(timeout=5) + except grpc.FutureTimeoutError: # pragma: no cover - guarded by socket check + ch.close() + pytest.skip(f"gRPC channel to {GRPC_TARGET} never became ready") + yield ch + ch.close() + + +@pytest.fixture +def stub(channel): + return pb_grpc.TestServiceStub(channel) + + +# --------------------------------------------------------------------------- # +# Tests +# --------------------------------------------------------------------------- # +def test_reflection_lists_service(channel): + """Server reflection advertises apidash.test.TestService.""" + reflection_pb2 = pytest.importorskip( + "grpc_reflection.v1alpha.reflection_pb2", + reason="grpcio-reflection not installed", + ) + reflection_pb2_grpc = pytest.importorskip( + "grpc_reflection.v1alpha.reflection_pb2_grpc", + reason="grpcio-reflection not installed", + ) + + refl_stub = reflection_pb2_grpc.ServerReflectionStub(channel) + request = reflection_pb2.ServerReflectionRequest(list_services="") + responses = refl_stub.ServerReflectionInfo(iter([request])) + + service_names = [] + for resp in responses: + service_names.extend(s.name for s in resp.list_services_response.service) + + assert "apidash.test.TestService" in service_names + + +def test_echo_unary(stub): + """Echo echoes the message back, with a server time and a sequence number.""" + resp = stub.Echo(pb.EchoRequest(message="hello-grpc"), timeout=10) + assert resp.message == "hello-grpc" + assert resp.server_time # non-empty ISO-8601 timestamp + assert resp.seq >= 1 + + +def test_get_random_user_populated(stub): + """GetRandomUser returns a fully populated mock user.""" + user = stub.GetRandomUser(pb.Empty(), timeout=10) + assert user.id + assert user.name + assert "@" in user.email + assert 18 <= user.age <= 80 + assert user.country + + +def test_stream_ticks_yields_n(stub): + """StreamTicks with count=N yields exactly N ticks, numbered 1..N.""" + n = 5 + ticks = list( + stub.StreamTicks(pb.TickRequest(count=n, interval_ms=10), timeout=30) + ) + assert len(ticks) == n + assert [t.seq for t in ticks] == list(range(1, n + 1)) + assert all(t.ts for t in ticks) + + +def test_sum_numbers_client_stream(stub): + """SumNumbers folds a stream of numbers into sum / count / average.""" + numbers = [1.5, 2.5, 4.0, 12.0] + + def gen(): + for value in numbers: + yield pb.NumberRequest(value=value) + + resp = stub.SumNumbers(gen(), timeout=10) + assert resp.count == len(numbers) + assert resp.sum == pytest.approx(sum(numbers)) + assert resp.average == pytest.approx(sum(numbers) / len(numbers)) + + +def test_chat_bidi_echoes_each_message(stub): + """Chat (bidi) echoes each message back, stamped with a server timestamp.""" + outgoing = [("alice", "hi"), ("bob", "hey"), ("alice", "bye")] + + def gen(): + for user, text in outgoing: + yield pb.ChatMessage(user=user, text=text) + + replies = list(stub.Chat(gen(), timeout=10)) + assert len(replies) == len(outgoing) + for (user, text), reply in zip(outgoing, replies): + assert reply.user == user + assert reply.text == text + assert reply.ts # server stamps the echo + + +def test_echo_metadata_request_and_response(stub): + """EchoMetadata reflects request metadata into the body AND returns + initial (x-server) + trailing (x-trailer) response metadata.""" + request_md = ( + ("authorization", "Bearer test-token"), + ("x-request-id", "abc123"), + ) + response, call = stub.EchoMetadata.with_call( + pb.Empty(), metadata=request_md, timeout=10 + ) + + echoed = dict(response.metadata) + assert echoed.get("authorization") == "Bearer test-token" + assert echoed.get("x-request-id") == "abc123" + + initial = dict(call.initial_metadata()) + assert initial.get("x-server") == "apidash-grpc-test" + + trailing = dict(call.trailing_metadata()) + assert trailing.get("x-trailer") == "ok" + + +def test_secure_echo_requires_credentials(stub): + """SecureEcho: no metadata -> UNAUTHENTICATED.""" + with pytest.raises(grpc.RpcError) as excinfo: + stub.SecureEcho(pb.EchoRequest(message="hi"), timeout=10) + assert excinfo.value.code() == grpc.StatusCode.UNAUTHENTICATED + + +def test_secure_echo_accepts_bearer_token(stub): + """SecureEcho: valid Bearer token -> success.""" + resp = stub.SecureEcho( + pb.EchoRequest(message="hi"), + metadata=(("authorization", "Bearer test-token"),), + timeout=10, + ) + assert "hi" in resp.message + assert resp.message.startswith("[authenticated]") + + +def test_secure_echo_accepts_api_key(stub): + """SecureEcho: valid x-api-key -> success.""" + resp = stub.SecureEcho( + pb.EchoRequest(message="viakey"), + metadata=(("x-api-key", "test-apikey"),), + timeout=10, + ) + assert "viakey" in resp.message + assert resp.message.startswith("[authenticated]") + + +def test_secure_echo_rejects_wrong_token(stub): + """SecureEcho: wrong credentials -> UNAUTHENTICATED.""" + with pytest.raises(grpc.RpcError) as excinfo: + stub.SecureEcho( + pb.EchoRequest(message="hi"), + metadata=(("authorization", "Bearer wrong"),), + timeout=10, + ) + assert excinfo.value.code() == grpc.StatusCode.UNAUTHENTICATED + + +def test_raise_error_maps_code(stub): + """RaiseError with code=5 fails with NOT_FOUND.""" + with pytest.raises(grpc.RpcError) as excinfo: + stub.RaiseError(pb.ErrorRequest(code=5, message="nope"), timeout=10) + assert excinfo.value.code() == grpc.StatusCode.NOT_FOUND diff --git a/mqtt/README.md b/mqtt/README.md new file mode 100644 index 0000000..0752e4b --- /dev/null +++ b/mqtt/README.md @@ -0,0 +1,27 @@ +# MQTT test rig + +A self-contained, local MQTT broker (Eclipse Mosquitto + a small test +publisher) for exercising API Dash's MQTT client -- QoS 1/2, retained messages, +wildcards, TLS, auth and MQTT v5. It is **not** part of the hosted Open Source +APIs; it only runs locally via Docker. + +## Setup & run the broker + +```bash +docker compose -f mqtt/docker-compose.yml up --build +``` + +Listeners: `1883` (plain, anonymous), `9001` (WebSocket), `8883` (TLS, +self-signed), `1884` (auth -- `testuser` / `testpass`). + +## Run the tests + +```bash +pip install -r mqtt/requirements-dev.txt +pytest mqtt/tests +``` + +The suite **skips gracefully** when paho-mqtt is missing or no broker is +reachable on `localhost:1883`. + +See [`docs/mqtt/`](../docs/mqtt/) for the per-feature pages. diff --git a/mqtt/requirements-dev.txt b/mqtt/requirements-dev.txt new file mode 100644 index 0000000..de5d19a --- /dev/null +++ b/mqtt/requirements-dev.txt @@ -0,0 +1 @@ +paho-mqtt>=2.0.0 # mqtt/tests: local MQTT broker round-trip tests (not needed by the prod app) diff --git a/tests/mqtt/test_mqtt.py b/mqtt/tests/test_mqtt.py similarity index 99% rename from tests/mqtt/test_mqtt.py rename to mqtt/tests/test_mqtt.py index 6c6fbb1..8caba29 100644 --- a/tests/mqtt/test_mqtt.py +++ b/mqtt/tests/test_mqtt.py @@ -12,7 +12,7 @@ so CI without a broker still passes. To run them for real: docker compose -f mqtt/docker-compose.yml up -d - pytest tests/mqtt/test_mqtt.py + pytest mqtt/tests The echo request->response test additionally needs the `publisher` service running; it skips (rather than fails) if no echo reply arrives. diff --git a/requirements-dev.txt b/requirements-dev.txt index 0b0ad87..65d0867 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -1,4 +1,3 @@ pytest httpx<0.28.0 pytest-asyncio -paho-mqtt>=2.0.0 # tests/mqtt: local MQTT broker round-trip tests (not needed by the prod app)