Qredence Documentation

Fleet RLM installs two equivalent entry points, fleet and fleet-rlm. Run them with uv run from the repository.

uv run fleet cli [databricks] [--mlflow local|databricks] [--session SESSION_ID] [--url URL]
                 [--mlflow-port 5001] [--mlflow-experiment NAME] [--startup-timeout 180]
uv run fleet web [--host 127.0.0.1] [--port 8080] [--public-url URL] [--session SESSION_ID] [--url URL]
                 [--mlflow local|databricks]
uv run fleet serve [--host 127.0.0.1] [--port 8000] [--mlflow local|databricks]
uv run fleet eval dataset|run|label ... [--mlflow local|databricks]
uv run fleet optimize --session SESSION_ID ... [--mlflow local|databricks]
uv run fleet trace verify TRACE_ID [--scope runtime|evaluation|optimization] [--mlflow local|databricks]

Invalid configuration, including an invalid flag value such as --port 0, exits with fleet: error: invalid configuration: … before Fleet starts anything.

MLflow backend

Every command accepts --mlflow local|databricks. Without the flag, Fleet uses [mlflow] mode from config/fleet.toml, which defaults to local. You never select the backend in .env. Environment variables override values only inside the selected backend.

Backend	Behavior
`local`	Traces go to `http://127.0.0.1:5001`. `fleet cli` starts this server if it isn’t running and stores data under `.fleet/mlflow`. Configure it in `[mlflow.local]`.
`databricks`	Traces go to Databricks managed MLflow and Unity Catalog trace tables. Fleet starts no tracking server. Configure it in `[mlflow.databricks]`.

uv run fleet cli databricks is shorthand for uv run fleet cli --mlflow databricks. Combining databricks with --mlflow local is an error. The --mlflow env option is retired.

`fleet cli`

Start the API server, wait for it to become ready, and open the terminal client. Fleet stops the services it started when you exit.

uv run fleet cli --session research-1

Flag	Default	Description
`databricks`	off	Positional. Use Databricks managed MLflow.
`--session`	`sess_` plus 8 random hex characters	Session ID to open or create. Use 1 to 64 letters, digits, dots, underscores, or hyphens.
`--url`	none	Connect to this server only. Fleet starts nothing.
`--mlflow-port`	`5001`	Local MLflow port. Rejected with the Databricks backend.
`--mlflow-experiment`	from config	Override the selected backend’s runtime experiment.
`--startup-timeout`	`180`	Seconds to wait for each service to become ready.

If a healthy API server already runs on the configured port, Fleet reuses it only when its MLflow backend, tracking URI, experiments, and trace location match. --url performs the same check. A mismatch aborts the connection. Stop the existing server or set a different FLEET_PORT.
If a service exits or misses --startup-timeout, the launch aborts and reports its log. Logs are .fleet/logs/server.log and .fleet/logs/mlflow.log.
On exit, Fleet stops the API server first to flush traces, then the MLflow server it owns. Services it reused keep running.
Ctrl+C during startup or in the client exits with status 130 after cleanup.

Setup checklist

When fleet cli starts its own server on a loopback address, it first checks your setup. It verifies four sections: Models, Database, MLflow, and Daytona. When every check passes without warnings, Fleet starts the services and opens the client. Otherwise it opens the setup checklist, where you fill in missing settings and save them to .env. See Setup checklist. When standard input isn’t a terminal, Fleet runs the checks without the checklist. If a check fails, it prints “Fleet setup needs attention” with each section’s status and exits with status 1. Run fleet cli in an interactive terminal to fix it. If setup can’t start, Fleet prints Could not complete setup and writes details to .fleet/logs/setup.log. Fleet skips the checklist when you pass --url or reuse a server that’s already running. See Use the terminal for slash commands.

`fleet web`

Serve the terminal client in a browser. Like fleet cli, it runs the setup checklist, starts the API server and local MLflow, and stops them when you exit. It then prints Fleet RLM Lite web app: <url>.

uv run fleet web --session research-1

Flag	Default	Description
`--host`	`127.0.0.1`	Web listener host.
`--port`	`8080`	Web listener port, from 1 to 65535.
`--public-url`	none	Browser-facing URL, for example behind a TLS proxy. Required when `--host` is a wildcard or IPv6 address.
`--session`	none	Use one Session ID for every browser client. Without it, each browser client gets its own Session.
`--url`	none	Connect to this API server only. Fleet starts nothing.
`--mlflow-port`	`5001`	Local MLflow port. Rejected with the Databricks backend.
`--mlflow-experiment`	from config	Override the selected backend’s runtime experiment.
`--startup-timeout`	`180`	Seconds to wait for each service to become ready.

To bind the web listener to a non-loopback host, set FLEET_ALLOW_NON_LOOPBACK_BIND=true and FLEET_WEB_AUTH_TOKEN to a secret of at least 32 non-whitespace characters. Fleet then accepts a terminal connection only when it carries an Authorization: Bearer <token> header, so put an authenticating proxy in front of the listener that adds the header. Every terminal connection must also come from the origin of --public-url, or of the listener address when you don’t set it.

`fleet serve`

Start only the FastAPI server.

uv run fleet serve [--host HOST] [--port PORT]

Flag	Default	Description
`--host`	`127.0.0.1`	Bind address. Also set by `FLEET_HOST` or `[server].host`.
`--port`	`8000`	Bind port, from 1 to 65535. Also set by `FLEET_PORT` or `[server].port`. Fleet rejects an invalid explicit value instead of using the default.

Fleet rejects a non-loopback bind address unless FLEET_ALLOW_NON_LOOPBACK_BIND=true. The API has no caller authentication, so expose it only behind a separately secured boundary. fleet serve doesn’t start MLflow. With the local backend, the server doesn’t start if MLflow is unreachable. Keep fleet cli running, or start MLflow with the same port and storage:

mkdir -p .fleet/mlflow
uv run mlflow server --host 127.0.0.1 --port 5001 \
  --backend-store-uri sqlite:///$PWD/.fleet/mlflow/mlflow.db \
  --default-artifact-root "$PWD/.fleet/mlflow/artifacts"

The eval, optimize, and trace commands also need a running tracking server. With the local backend, they exit with an error if it’s unreachable.

`fleet eval`

Create evaluation datasets, label traces, and score live runs.

`fleet eval dataset`

Create the synthetic evaluation dataset.

Flag	Default	Description
`--name`	empty	Dataset name. With Databricks, use `catalog.schema.table`.
`--seed`	`7`	Random seed.
`--long-context`	off	Add synthetic 20K to 200K character log cases.

Fleet RLM 0.8.1 removed the --oolong and --longcot flags. Oolong-synth and LongCoT benchmarks now run from development notebooks in a Fleet RLM source checkout. Their traces stay out of the curated optimization dataset.

`fleet eval run`

Run synthetic cases against a live server and score them. The server must trace into the selected evaluation experiment.

Flag	Default	Description
`--url`	`http://{host}:{port}` from config	Server to run against.
`--seed`	`7`	Random seed.
`--limit N`	`0`	Run only the first N cases. `0` runs all.
`--collect-only`	off	Run and label the Turns, but skip the MLflow judge suite.

`fleet eval label`

Write gold labels onto a trace. Partial updates keep existing expectations.

uv run fleet eval label TRACE_ID --answer "Reviewed answer" --value "42" --route rlm

Flag	Description
`--answer`	Expected answer.
`--value`	Value that must appear in the answer. Repeatable.
`--route`	Expected route: `direct`, `flex`, or `rlm`.
`--expectations-file`	Replace structured artifact and delegation expectations with a JSON object.
`--critical`, `--no-critical`	Pin this example to GEPA holdout and require a perfect candidate score. Requires at least one expected value.
`--status`	Review status. New labels default to `approved`. Updates keep the existing status.
`--group`	Source group for train, dev, and holdout splits. Defaults to the existing label, then the trace’s Session ID.
`--reviewer`	Reviewer name. Defaults to `human`.

`fleet optimize`

Evaluate a Session’s reviewed examples and compile a GEPA candidate. This command calls live models and Daytona.

uv run fleet optimize --session research-1 --gepa-budget light
uv run fleet optimize --session research-1 --approve CANDIDATE_VERSION

Flag	Default	Description
`--session`	required	Session whose engine serves an approved candidate.
`--gepa-budget`	`light`	`light`, `medium`, or `heavy`.
`--max-metric-calls N`	none	GEPA search metric-call budget. Replaces `--gepa-budget`.
`--threads`	`4`	Parallel threads, from 1 to 32.
`--validation-ratio`	`0.35`	Share of examples in the dev split, from 0.1 to 0.5.
`--reflection-minibatch-size N`	`3`	Examples per GEPA reflection step, at least 1.
`--max-candidate-proposals N`	none	Stop the search after this many candidate proposals.
`--trace-search`	off	Also trace search rollouts in MLflow.
`--approve VERSION`	none	Serve a saved, passing candidate from the next Turn. Fleet checks the candidate’s current evidence first.

Optimization needs at least eight approved examples from three source groups, captured with replay enabled, including at least one critical holdout example (fleet eval label ... --critical). A passing candidate stays inactive until you approve it. --max-metric-calls bounds the search only. Reflection, judging, and the three final validation passes are additional work. A candidate covers the Turn program’s instructions and the instructions Fleet gives child agents, with separate policies for children that can delegate further and children that can’t. Runtime limits and tool allowlists still come from your configuration, not the candidate. Each rollout runs in its own isolated Sandbox. If Fleet can’t confirm that a rollout’s Sandbox was cleaned up, it stops the run and treats its evidence as invalid. To run optimization on a running server instead, use /optimize in the terminal or the experiments routes.

`fleet trace verify`

Flush pending trace exports and confirm that one stored trace is readable with its spans.

uv run fleet trace verify TRACE_ID --scope runtime

Flag	Default	Description
`TRACE_ID`	required	Trace ID. For Unity Catalog, use the fully qualified `trace:/` reference.
`--scope`	`runtime`	`runtime`, `evaluation`, or `optimization` experiment.

With the Databricks backend, the command also checks that the experiment is bound to the configured Unity Catalog schema and table prefix.

Fleet-RLM

Fleet Prime Agent

Fleet Pi

GEPA Omni

Qredence Skills

Fleet Reasoner

Qredence Plugins

CLI reference

MLflow backend

`fleet cli`

Setup checklist

`fleet web`

`fleet serve`

`fleet eval`

`fleet eval dataset`

`fleet eval run`

`fleet eval label`

`fleet optimize`

`fleet trace verify`

​MLflow backend

​fleet cli

​Setup checklist

​fleet web

​fleet serve

​fleet eval

​fleet eval dataset

​fleet eval run

​fleet eval label

​fleet optimize

​fleet trace verify

MLflow backend

`fleet cli`

Setup checklist

`fleet web`

`fleet serve`

`fleet eval`

`fleet eval dataset`

`fleet eval run`

`fleet eval label`

`fleet optimize`

`fleet trace verify`