gchq/Bailo

About The Project

Bailo helps you manage the lifecycle of machine learning to support scalability, impact, collaboration, compliance and sharing.

Getting Started

Running Locally with Docker

Requirements

Follow the instructions below to get a local development environment running. You will need:

• Node v26
• Docker / Docker Compose

Development Build

To run in development mode (modified files on your host machine will be reloaded into the running application):

[!NOTE] Ensure Docker and Docker Compose are installed before running these commands.

git clone https://github.com/gchq/Bailo.git && cd Bailo
npm install
npm run certs

# This builds all the Bailo images, rerun it when you update dependencies.
docker compose build --parallel

# Then run the development instance of Bailo.
docker compose up -d

On first run, it may take a while (up to 30 seconds) to start up. It needs to build several hundred TypeScript modules. These are cached however, so future starts only require a few seconds. You should access the site via localhost:8080.

The registry requires a JWKS file for the token authentication with the backend application. For development, a JWKS file is generated by running npm run certs. For production, the script generateJWKS.ts can be used to generate a JWKS file for the public key referenced in the backend application configuration.

To stop the running services:

docker compose down

Production Build

The above docker compose commands can, alternatively, be run with -f compose.prod.yml to have a more static version of the application running which mimics a production environment. Volumes are not dynamically mounted, the build time is slower as the frontend is pre-compiled, and environment variables & users are more production ready. This build is also more reliable for our cypress tests:

# Build and run the productionised instance of Bailo.
docker compose -f compose.yaml -f compose.prod.yaml --env-file  prod.env up --force-recreate --build  -d

# Stop the service and destroy the volumes
docker compose -f compose.yaml -f compose.prod.yaml down -v

The above docker compose approaches are preferred, however we also provide a monolithic Dockerfile.standalone. To run in standalone mode, not development mode (http://localhost:8080). Not for production use:

docker build -t "bailo:standalone" -f ./Dockerfile.standalone --build-context python=./lib/python .
docker run --name bailo -p 8080:8080 -d bailo:standalone

Kubernetes Deployment

We recommend using our helm charts to deploy Bailo to Kubernetes.

We expect the administrator to provide their own forms of authentication. By default all users authenticate as 'user'.

Testing a New Deployment

You can test out your new deployment using the example models which can be found in frontend/cypress/fixtures minimal_binary.zip and minimal_code.zip. There are also example forms in the scripts folder minimal_model_schema.json, minimal_data_card_schema.json and minimal_access_request_schema.json.

Logical Project Flow (Overview)

Components

NGINX
UI
API
Docker Registry
Mongo
S3

1. A user accesses a URL. We use NextJS routing to point it to a file in frontend/pages. [xxx].tsx files accept any route, xxx.tsx files allow only that specific route.
2. Data is loaded using SWR. Data loaders are stored in frontend/actions. Each one exposes variables to specify if it is loading, errored, data, etc.
3. Requests to the backend get routed through express within backend/src/routes.ts. Each route is an array with all items being middleware except the last, which is the handler ([...middleware, handler]).
4. Routes interact with the database via mongoose, which stores models in backend/src/models.

Stroom Integration

Bailo integrated with Stroom, a GCHQ-developed data storage, processing and analysis platform.

This integration enables Bailo to create and send event logging to Stroom for ingestion and later analysis, creating an audit trail and ensuring health within a given Bailo deployment. The integration is optional.

Known Issues

• Issue: Sometimes Docker struggles when you add a new dependency.
  Fix: Run docker compose down --rmi all followed by docker compose up --build.

• Issue: Sometimes SWR fails to install its own binary and the project will refuse to start up (development only)
  Fix: Run npm uninstall next && npm install next. Some users report still having issues. If so, run: rm -rf node_modules && rm -rf package-lock.json && npm cache clean -f && npm i.

• Issue: Unable to authenticate to the Docker registry / compile binaries.
  Fix: Make sure that your authentication proxy is setup to allow the 'Authorisation' header. Make sure that your application is able to access the Docker registry internally as it will not provide user authentication.

• Issue: npm run build fails in the frontend directory with Error: EACCES: permission denied .
  Fix: Run sudo rm -rf .next in the frontend directory. This typically happens because docker compose has modified the mounted volume owner.

Usage

See our user documentation

Contributing

If you'd like to make a contribution then the details for doing all of that are in CONTRIBUTING.md.

License

Bailo is released under the Apache 2.0 Licence and is covered by Crown Copyright. See LICENSE.txt for more information.

Acknowledgments

• Othneils's README Template
• Stroom's Code Contribution Guidelines