Advaita Drive (UI-branded DataHub) is a file-storage web application for end users of iPathwayGuide (iPG) and iScanGuide (iSG). It runs on CoreWeave Kubernetes from ~/working/advaita-deploy/projects/advaita-drive as a Kustomize tree with qa and prod overlays over a shared base. The application source is advaita-corporation/advaita-drive-service.
These notes are distilled from advaita-drive-service develop (@ 326b135) and the deploy tree on feature/advaita-drive-deploy-scripts, as of 2026-07-12. Provenance is unverified against the running clusters.
Drive gives users a Google-Drive-like place to store large research data files (FASTQ sequencing reads and similar) in CoreWeave object storage. It presents folders and files through a single-page UI at /ui/* and a REST API at /api/*. Files are organized per-user, fenced per-org (multi-tenant), and encrypted at rest.
iPG and iSG users need durable storage for genomics-scale inputs and outputs that is addressable from the rest of the Advaita suite. Drive is that store, sitting alongside the compute products under the same identity (az2) and object-storage backend (CoreWeave).
The defining property is that large file bytes never transit the pod. Uploads and downloads go browser-direct to CoreWeave S3 using presigned multipart URLs the backend mints; the service holds metadata, auth, and orchestration only. State is Postgres (the shared files schema) plus object storage, so the workload is stateless and runs two replicas.
drive_file row.orgId.pipeline_job row, a poller claims it (SELECT … FOR UPDATE SKIP LOCKED) and calls the datahub-launcher, and a token-authenticated callback records the QC outputs as sibling drive_file rows. The launcher is the separate wisp / advaita-service-foxfire service; the Nextflow driver images are nf-fastqc-driver and siblings.Two environments over one base. The image is 789905347053.dkr.ecr.us-west-2.amazonaws.com/advaita/drive, pinned per-deploy from ECR by the deploy scripts (never committed to git). The container listens on 8081; a ClusterIP Service exposes port 80.
| Env | Namespace | Cluster | Node pool | Primary host |
|---|---|---|---|---|
| qa | drive-qa |
useast13-dev |
barad-dur |
datahub-bd.cw.advaitabio.com |
| prod | drive |
useast13-prod |
general-purpose-amd-turin |
datahub.advaitabio.com |
Each ingress also carries the legacy drive-qa.cw / drive.cw host as a second rule for the domain cutover. Ingress is nginx with cert-manager TLS via the namespace-scoped Issuer letsencrypt-prod-route53 (DNS-01 against Route53). / redirects to /ui/ at the ingress; proxy-body-size: 0.
Base workload: replicas: 2, RollingUpdate maxUnavailable=0, PodDisruptionBudget minAvailable=1, non-root runAsUser 1000, read-only root filesystem with a writable /tmp emptyDir, drop: ALL capabilities, seccomp RuntimeDefault. A projected ServiceAccount token (audience sts.amazonaws.com) is mounted for AWS web-identity auth to KMS — CoreWeave has no pod-identity webhook, so credentials are wired explicitly.
Deploy scripts live in projects/advaita-drive/scripts/: deploy.sh --env=qa and deploy-prod.sh discover a tag from ECR, inject it at apply time, apply the overlay, and restore the file. setup.sh --env=<env> is the first-time bootstrap (namespace, bucket + CORS, ECR pull secret, KMS CMK + role via ensure_kms, secrets).
Non-secret config is k8s/overlays/<env>/config.env → ConfigMap advaita-drive-config (hash-suffixed, so edits trigger a rollout). Secrets are secrets.yaml (gitignored; a secrets.yaml.example template is tracked) → Secret advaita-drive-secrets, carrying the Postgres password, CoreWeave S3 keys, and the az2 client secret. Config keys of note: DRIVE_STORAGE_S3_* (CoreWeave bucket + endpoint), DRIVE_AUTH_AZ2_* (introspection base URL, client id, redirect uri), SPRING_DATASOURCE_* (Postgres, currentSchema=files), DRIVE_ENCRYPTION_* (KMS CMK alias + region), AWS_ROLE_ARN (the per-env web-identity role).
| Dependency | Use | Endpoint / id |
|---|---|---|
| az2 | OAuth2 introspection + token exchange | qa barad-dur.advaitabio.com/oauth-provider; prod apps.advaitabio.com/oauth-provider |
| CoreWeave object storage | file blobs + sidecars | cwobject.com; buckets advaita-drive-qa / advaita-drive-prod |
Postgres (files schema) |
metadata, upload sessions, pipeline jobs | qa clouddev…; prod prod-cloud… RDS us-west-2 |
| AWS KMS | per-tenant key wrapping | CMK alias/advaita-datahub-qa / -prod, us-west-2 |
| datahub-launcher (wisp) | FastQC pipeline launch + callback | DRIVE_PIPELINE_LAUNCHER_URL |
iPG and iSG end users through the browser. A future service-to-service download path (outbound pre-processing, per the roadmap) would add sibling-service callers.
File browsing, upload initiation, and download-URL minting stop (the control plane is fully user-facing). In-flight browser↔S3 transfers already presigned continue until their URL expires. Blobs and their KMS-wrapped keys are unaffected — the DR runbook can rebuild drive_file rows from object storage plus the CMK.
advaita-corporation/advaita-drive-service — a single Gradle multi-project (Kotlin 2.0.20, JDK 21) producing one Spring Boot 3.3.4 fat JAR that serves both the API and the bundled UI. Backend: Spring Web MVC, Security, Data JPA, Flyway (V1–V13), Caffeine, AWS SDK v2 (S3 presign + KMS). Frontend: Compose HTML (Kotlin/JS via Kotlin Multiplatform), compiled into the backend's static/ui/. A shared module holds the serialized DTOs. Image: Temurin 21 JRE Alpine, USER 1000. Version 0.1.0-SNAPSHOT; deploy scripts tag an immutable <version>-<sha> for the running image.
Build and deploy currently run from the advaita-deploy bash scripts (ECR tag pick → kustomize apply → rollout). The target is the Vivarta pipeline the other products are moving to: a projects/advaita-drive/vivarta.yml for the two-stage docker_build to ECR, and a kustomize/kubectl deploy boundary (set image tag → apply overlay → rollout), mirroring the iSCanGuide port. See advaita-product-iscanguide Migration section for the reference shape.
Authorization and tenancy:
DELETE /files/{id} fences on orgId but not ownerId — any authenticated user can delete another user's files/folders in the same org (recursive for folders). Every read path fences on both; delete is the gap.FastqcOutputIngestService) uses the client-supplied outputKeys verbatim as the new row's storage key, so a caller holding a job's runId + callbackToken can register a row pointing at another org's object. Validate each key sits under the job's own prefix.DRIVE_OPS_ADMIN_EMAILS, /api/admin/files/{id}/content) is enabled in the qa config.env but lives on the unmerged branch feature/ops-admin-file-download. The env var is inert until that branch merges.NoAuthMode's production guard is a blocklist of exact profile names; an unnamed or mis-named profile boots no-auth as the real advaita org. Deny-by-default (require an explicit dev profile to enable) is safer.@CrossOrigin(origins = ["*"]) on the file API; anyRequest().permitAll() default in SecurityConfig.Pipeline correctness (two replicas):
PipelineJobPoller.expireStale() is an unlocked read-modify-write over a full-table scan, running on both replicas and racing the callback — a finished job can be clobbered to FAILED. The claim path (SKIP LOCKED) is correct; the expiry sweep is not.SELECT FOR UPDATE/@Version; concurrent at-least-once redeliveries can double-ingest output rows.runId until the 30-minute expiry; LauncherClient has no timeout or retry and blocks a scheduler thread (pool of 4) on a hung launcher.pipeline_job.file_id — findByFileIdIn runs on every file listing.Storage / lifecycle:
FileService.create writes the sidecar at CREATED time, so aborted uploads leave ghost CREATED rows (visible in listings) and orphan sidecars that are never reaped.sizeBytes becomes the authoritative file size and the download Content-Length, never reconciled against the declared size or the object's actual length.Content-Disposition is built from an unsanitized filename (path separators stripped, quotes not) — a " in a filename breaks out of the header value. Use RFC 5987 filename*.Key management (deploy-repo + AWS):
ScheduleKeyDeletion/DisableKey, a key-policy deny except break-glass, no alias repoint without re-wrap — are a prod go-live prerequisite and are not built. See advaita-key-management-standard.Deploy manifests:
/actuator/health (includes DB/S3), so a downstream blip can restart both replicas; use /actuator/health/liveness.base/deployment.yaml and serviceaccount.yaml describe an images: transformer and encryption OFF that no longer match the overlays/config; service.yaml and overlay headers still name the pre-cutover drive-*.cw hosts as primary; overlay headers reference overlays/<env>/setup.sh where the script is scripts/setup.sh --env=<env>.Roadmap (designed, not built):
scanStatus = SKIPPED).files schema is provisionally owned here; alignment with the shared schema owned by the files / vcf-files services is planned.