# Node HTTP Servers for Testing

This page describes the Node.js-based HTTP server implementation located in `netwerk/test/httpserver/NodeServer.sys.mjs`. This system provides HTTP, HTTPS, HTTP/2, HTTP/3, WebSocket, and proxy servers for use in xpcshell tests.

## Overview

The NodeServer system allows tests to spawn Node.js-based HTTP servers that run in separate processes. Unlike the JavaScript-based httpd.sys.mjs server, these Node servers provide full support for modern protocols like HTTP/2, HTTP/3, WebSockets over HTTP/2, and various proxy configurations.

## Architecture

The NodeServer system consists of three main components:

### 1. NodeServer.sys.mjs (Test Interface)

This is the Firefox/XPCShell side interface that tests use to control Node servers. It provides:

- Server classes for different protocols (HTTP, HTTPS, HTTP/2, WebSocket, Proxies)
- Methods to start/stop servers
- Methods to execute code in the Node.js context
- Methods to register request handlers

### 2. runxpcshelltests.py (Test Harness)

The xpcshell test harness automatically starts the moz-http2.js server when tests run:

- Spawns `node moz-http2.js` as a subprocess
- Sets the `MOZNODE_EXEC_PORT` environment variable with the server's HTTP port
- Handles server lifecycle (startup/shutdown)

### 3. moz-http2.js (Node Server)

This is the main Node.js HTTP/2 server that:

- Listens on the port specified in `MOZNODE_EXEC_PORT`
- Handles test requests and DNS resolution
- Provides special endpoints for process management:
  - `/fork` - Spawns a new Node.js child process
  - `/execute/{id}` - Executes code in a forked process
  - `/kill/{id}` - Terminates a forked process
  - `/forkH3Server` - Spawns an HTTP/3 server

## How It Works

### Server Startup Flow

```
runxpcshelltests.py
    |
    v
Spawns node process: node moz-http2/moz-http2.js
    |
    v
Sets MOZNODE_EXEC_PORT environment variable
    |
    v
moz-http2.js server starts listening on random port
    |
    v
Tests can now use NodeServer.sys.mjs to create servers
```

### Process Forking Flow

When a test creates a server (e.g., `new NodeHTTPServer()`):

```
Test calls server.start()
    |
    v
NodeServer.fork() sends POST to http://127.0.0.1:{MOZNODE_EXEC_PORT}/fork
    |
    v
moz-http2.js receives /fork request
    |
    v
Calls fork() to spawn moz-http2-child.js
    |
    v
Returns unique process ID to test
    |
    v
Test uses NodeServer.execute(id, code) to run code in child process
    |
    v
Code is sent via POST to /execute/{id}
    |
    v
moz-http2.js forwards code to child process via IPC
    |
    v
moz-http2-child.js receives message, runs eval(code)
    |
    v
Result is sent back through IPC chain to test
```

### Code Execution in Child Process

The child process (moz-http2-child.js) is extremely simple:

```javascript
process.on("message", msg => {
  const code = msg.code;
  let evalResult = eval(code);  // Execute the code
  process.send({ result: evalResult });  // Send result back
});
```

This allows tests to:

1. Define classes and functions in the Node.js context
2. Start HTTP servers
3. Register request handlers
4. Query server state

## Server Types

### NodeHTTPServer

Basic HTTP/1.1 server.

```javascript
const { NodeHTTPServer } = ChromeUtils.importESModule(
  "resource://testing-common/NodeServer.sys.mjs"
);

let server = new NodeHTTPServer();
await server.start(); // Random port
const port = server.port();
const origin = server.origin(); // http://localhost:{port}

// Register a path handler
await server.registerPathHandler("/test", (req, resp) => {
  resp.writeHead(200);
  resp.end("Hello World");
});

// When done
await server.stop();
```

### NodeHTTPSServer

HTTPS server using HTTP/1.1.

```javascript
const { NodeHTTPSServer } = ChromeUtils.importESModule(
  "resource://testing-common/NodeServer.sys.mjs"
);

let server = new NodeHTTPSServer();
await server.start(8443); // Specific port, or 0 for random
// Uses certificate from netwerk/test/unit/http2-cert.pem
```

### NodeHTTP2Server

HTTP/2 over TLS server.

```javascript
const { NodeHTTP2Server } = ChromeUtils.importESModule(
  "resource://testing-common/NodeServer.sys.mjs"
);

let server = new NodeHTTP2Server();
await server.start();
// Supports HTTP/2 specific features like server push, multiplexing

// Check session count
let count = await server.sessionCount();
```

### HTTP/3 Server

HTTP/3 (QUIC) server.

```javascript
const { HTTP3Server } = ChromeUtils.importESModule(
  "resource://testing-common/NodeServer.sys.mjs"
);

let server = new HTTP3Server();
let path = "/path/to/http3/server/binary";
let dbPath = "/path/to/quic/database";
await server.start(path, dbPath);
const port = server.port();
const masquePort = server.masque_proxy_port();
```

### NodeWebSocketServer

WebSocket server over HTTPS.

```javascript
const { NodeWebSocketServer } = ChromeUtils.importESModule(
  "resource://testing-common/NodeServer.sys.mjs"
);

let server = new NodeWebSocketServer();
await server.start();

// Register custom message handler
await server.registerMessageHandler((data, ws) => {
  ws.send("Echo: " + data);
});
```

### NodeWebSocketHttp2Server

WebSocket over HTTP/2 (RFC 8441).

```javascript
const { NodeWebSocketHttp2Server } = ChromeUtils.importESModule(
  "resource://testing-common/NodeServer.sys.mjs"
);

let server = new NodeWebSocketHttp2Server();
await server.start(0, false); // port, fallbackToH1
```

### Proxy Servers

```javascript
const { NodeHTTPProxyServer, NodeHTTPSProxyServer, NodeHTTP2ProxyServer } =
  ChromeUtils.importESModule("resource://testing-common/NodeServer.sys.mjs");

// HTTP proxy
let httpProxy = new NodeHTTPProxyServer();
await httpProxy.start();

// HTTPS proxy
let httpsProxy = new NodeHTTPSProxyServer();
await httpsProxy.start();

// HTTP/2 proxy
let http2Proxy = new NodeHTTP2ProxyServer();
await http2Proxy.start(0, true, 100); // port, auth, maxConcurrentStreams
```

## Advanced Usage

### Registering Path Handlers

Path handlers are functions that process requests for specific paths:

```javascript
await server.registerPathHandler("/api/data", (req, resp) => {
  // req is Node's http.IncomingMessage
  // resp is Node's http.ServerResponse

  resp.setHeader("Content-Type", "application/json");
  resp.writeHead(200);
  resp.end(JSON.stringify({ status: "ok" }));
});
```

### Executing Arbitrary Code

You can execute any JavaScript code in the Node.js context:

```javascript
// Define a function
await server.execute(`
  function customHandler(req, resp) {
    resp.writeHead(200);
    resp.end("Custom response");
  }
`);

// Use the function
await server.execute(`global.path_handlers["/custom"] = customHandler`);

// Query state
let result = await server.execute(`Object.keys(global.path_handlers).length`);
```

### Passing Functions

You can pass JavaScript functions directly:

```javascript
function myHandler(req, resp) {
  resp.writeHead(200);
  resp.end("Handler from test");
}

// The function is serialized and defined in the Node context
await server.execute(myHandler);

// Now call it
await server.execute(`myHandler(someReq, someResp)`);
```

### Working with Global State

The Node.js child processes maintain global state:

```javascript
// Set up global variables
await server.execute(`global.requestCount = 0;`);

// Use in handlers
await server.registerPathHandler("/count", (req, resp) => {
  global.requestCount++;
  resp.writeHead(200);
  resp.end(`Request ${global.requestCount}`);
});

// Query state
let count = await server.execute(`global.requestCount`);
```

## Android Support

The system includes ADB port forwarding support for Android testing:

```javascript
// Automatically handled when MOZ_ANDROID_DATA_DIR is set
// The ADB class in NodeServer.sys.mjs forwards ports using:
// adb reverse tcp:{port} tcp:{port}
```

This means xpcshell-tests on Android can pretend to connect to `localhost:${port}` while the node server actually runs on the host.

## Certificate Handling

HTTPS and HTTP/2 servers automatically install test certificates:

- Certificate: `netwerk/test/unit/http2-cert.pem`
- CA: `netwerk/test/unit/http2-ca.pem`
- Key: `netwerk/test/unit/http2-cert.key`

Proxy servers use different certificates:

- Certificate: `netwerk/test/unit/proxy-cert.pem`
- CA: `netwerk/test/unit/proxy-ca.pem`
- Key: `netwerk/test/unit/proxy-cert.key`

To skip automatic certificate installation:

```javascript
let server = new NodeHTTPSServer();
server._skipCert = true;
await server.start();
```

The certificates are valid for the following domains: `localhost`, `foo.example.com`, `alt1.example.com`, `alt2.example.com`
Check `http2-cert.pem.certspec` and `proxy-cert.pem.certspec` for the up to date information.

If you need the certs to be valid for more domains, consider using:
```javascript
const certOverrideService = Cc[
  "@mozilla.org/security/certoverride;1"
].getService(Ci.nsICertOverrideService);
certOverrideService.setDisableAllSecurityChecksAndLetAttackersInterceptMyData(true);
```

## Best Practices

### Always Stop Servers

Always stop servers in cleanup to avoid resource leaks:

```javascript
registerCleanupFunction(async () => {
  await server.stop();
});
```

### Use Random Ports

Use port 0 (or omit the port parameter) to get a random available port:

```javascript
await server.start(); // Random port
// NOT: await server.start(8080); // Fixed port causes conflicts
```

### Helper Function for Multiple Server Types

Use the `with_node_servers` helper to test multiple server types:

```javascript
const { with_node_servers, NodeHTTPServer, NodeHTTP2Server } =
  ChromeUtils.importESModule("resource://testing-common/NodeServer.sys.mjs");

await with_node_servers(
  [NodeHTTPServer, NodeHTTP2Server],
  async server => {
    // This runs once for each server type
    let response = await fetch(server.origin() + "/test");
    // ... test code ...
  }
);
// Servers are automatically stopped
```

### Error Handling

Wrap server operations that may fail in try-catch blocks:

```javascript
try {
  await server.execute(`
    global.server.listen(port);
  `);
} catch (e) {
  // Handle execution errors
  console.error("Server setup failed:", e);
}
```

### Debugging

To debug issues, you can inspect the Node.js process:

```javascript
// Log in Node context
await server.execute(`console.log("Debug info:", someVariable)`);

// Check the xpcshell test output for Node.js console.log output
```

## Example Tests

### Simple HTTP Server Test

```javascript
add_task(async function test_simple_http_server() {
  let server = new NodeHTTPServer();
  await server.start();

  registerCleanupFunction(async () => {
    await server.stop();
  });

  await server.registerPathHandler("/hello", (req, resp) => {
    resp.writeHead(200, { "Content-Type": "text/plain" });
    resp.end("Hello, World!");
  });

  let response = await fetch(server.origin() + "/hello");
  let text = await response.text();
  Assert.equal(text, "Hello, World!");
});
```

### HTTP/2 Server Test

```javascript
add_task(async function test_http2_multiplexing() {
  let server = new NodeHTTP2Server();
  await server.start();

  registerCleanupFunction(async () => {
    await server.stop();
  });

  await server.registerPathHandler("/data", (req, resp) => {
    resp.writeHead(200);
    resp.end("data");
  });

  // Make multiple requests
  let responses = await Promise.all([
    fetch(server.origin() + "/data"),
    fetch(server.origin() + "/data"),
    fetch(server.origin() + "/data"),
  ]);

  // All requests should use the same HTTP/2 session
  let sessionCount = await server.sessionCount();
  Assert.equal(sessionCount, 1, "Should reuse single HTTP/2 session");
});
```

### WebSocket Test

```javascript
add_task(async function test_websocket() {
  let server = new NodeWebSocketServer();
  await server.start();

  registerCleanupFunction(async () => {
    await server.stop();
  });

  await server.registerMessageHandler((data, ws) => {
    ws.send("Echo: " + data);
  });

  let wsc = new WebSocketConnection();
  await wsc.open(server.origin().replace("https", "wss") + "/");
  wsc.send("test message");

  let messages = await wsc.receiveMessages();
  Assert.equal(messages[0], "Echo: test message");

  wsc.close();
  await wsc.finished();
});
```

### Proxy Test

```javascript
add_task(async function test_http_proxy() {
  let proxy = new NodeHTTPProxyServer();
  await proxy.start();

  registerCleanupFunction(async () => {
    await proxy.stop();
  });

  // Proxy filter is automatically registered
  // All HTTP requests will now go through the proxy

  let response = await fetch("http://example.com/");
  Assert.equal(response.status, 200);
});
```

### Async State Management Test

This test demonstrates concurrent async operations with proper result routing:

```javascript
add_task(async function test_async_state_management() {
  let server = new NodeHTTP2Server();
  await server.start();
  registerCleanupFunction(async () => {
    await server.stop();
  });

  // Initialize state in the Node.js context
  await server.execute(`global.asyncResults = [];`);

  // Define an async function that takes time to complete
  await server.execute(`
    global.asyncCounter = 0;
    global.performAsyncOperation = function(delay, value) {
      return new Promise(resolve => {
        setTimeout(() => {
          global.asyncCounter++;
          global.asyncResults.push({ counter: global.asyncCounter, value });
          resolve({ counter: global.asyncCounter, value });
        }, delay);
      });
    };
  `);

  // Launch two concurrent async operations with different delays
  let op1 = server.execute(`performAsyncOperation(100, "first")`);
  let op2 = server.execute(`performAsyncOperation(50, "second")`);

  // Wait for both to complete
  let result1 = await op1;
  let result2 = await op2;

  // op2 completes first (50ms delay) so it gets counter=1
  equal(result2.counter, 1);
  equal(result2.value, "second");

  // op1 completes second (100ms delay) so it gets counter=2
  equal(result1.counter, 2);
  equal(result1.value, "first");

  // Verify the global state was updated correctly
  let results = await server.execute(`global.asyncResults`);
  equal(results.length, 2);
  equal(results[0].value, "second");  // First to complete
  equal(results[1].value, "first");   // Second to complete

  let counter = await server.execute(`global.asyncCounter`);
  equal(counter, 2);

  await server.stop();
});
```

This test demonstrates:
- Multiple concurrent `execute()` calls on the same server
- Each operation receives its correct result despite different completion times
- Global state is properly shared across executions
- The message handler system correctly routes responses to their respective promises

## Common Pitfalls

### Not Awaiting Async Operations

All server operations are asynchronous:

```javascript
// WRONG
server.start();
server.registerPathHandler("/test", handler);

// CORRECT
await server.start();
await server.registerPathHandler("/test", handler);
```

### Forgetting to Stop Servers

Servers must be explicitly stopped:

```javascript
// WRONG
add_task(async function test() {
  let server = new NodeHTTPServer();
  await server.start();
  // ... test code ...
  // Server is never stopped!
});

// CORRECT
add_task(async function test() {
  let server = new NodeHTTPServer();
  await server.start();
  registerCleanupFunction(async () => {
    await server.stop();
  });
  // ... test code ...
});
```

### Hardcoded Ports

Avoid hardcoded ports as they can cause conflicts when tests run in parallel:

```javascript
// WRONG
await server.start(8080);

// CORRECT
await server.start(); // or await server.start(0);
let port = server.port();
```

### Scope Issues in Handlers

Remember that handlers run in the Node.js context, not the test context:

```javascript
// WRONG - testVariable is not accessible in Node.js
let testVariable = "value";
await server.registerPathHandler("/test", (req, resp) => {
  resp.end(testVariable); // ERROR: testVariable is undefined
});

// CORRECT - Pass values explicitly
let testVariable = "value";
await server.execute(`global.sharedValue = "${testVariable}"`);
await server.registerPathHandler("/test", (req, resp) => {
  resp.end(global.sharedValue);
});
```

## Implementation Details

### Process IDs

When you call `NodeServer.fork()`, the moz-http2.js server generates a random 6-character process ID. This ID is used to route commands to the correct child process.

### Communication Protocol

Communication uses HTTP POST requests with JSON payloads:

```
POST /execute/{processId}
Body: JavaScript code to execute

Response: { "result": <return value>, "error": "", "errorStack": "", "messageId": <id> }
```

### Message Handler System

The system uses a message handler architecture to support concurrent async operations:

1. Each `/execute/{processId}` request generates a unique 6-character `messageId`
2. A promise handler is stored in `forked.messageHandlers[messageId] = { resolve, reject }`
3. The `messageId` is sent to the child process along with the code
4. The child process returns the result with the same `messageId`
5. The response is routed to the correct promise handler using the `messageId`

This design allows multiple async operations to run concurrently on the same child process without interfering with each other. For example, you can call `server.execute()` multiple times in parallel and each will properly receive its own result.

### Eval-based Execution

Code execution uses `eval()` in the child process:

```javascript
// In moz-http2-child.js
process.on("message", msg => {
  const code = msg.code;
  const messageId = msg.messageId;
  let evalResult = eval(code);
  if (evalResult instanceof Promise) {
    evalResult
      .then(x => process.send({ result: x, messageId }))
      .catch(e => process.send({ error: e.toString(), messageId }));
  } else {
    process.send({ result: evalResult, messageId });
  }
});
```

This allows executing:

- Variable declarations
- Function definitions
- Expressions
- Async operations (Promise returns are handled automatically)
- Concurrent async operations without conflicts

### Function Serialization

When you pass a function to `execute()`, it's serialized:

```javascript
// You pass:
function handler(req, resp) { resp.end("ok"); }

// The system sends:
"handler = function handler(req, resp) { resp.end(\"ok\"); };"
```

## See Also

- `netwerk/test/httpserver/nsIHttpServer.idl` - JavaScript HTTP server
- `testing/xpcshell/moz-http2/moz-http2.js` - Node HTTP/2 server implementation
- `netwerk/test/unit/` - Example tests using NodeServer
- `netwerk/docs/http_server_for_testing.rst` - JavaScript-based httpd.sys.mjs server