# DSP Platform Docker Setup

This repository now includes a Docker-based development environment that brings up:

- **PHP + Apache** web server (with Rscript available for the automated analyses)
- **MySQL 8.0** database seeded with the `db/niph_dsps.sql` dump on first run
- **phpMyAdmin** for administering the database through the browser
- **JupyterHub (per-user R-enabled JupyterLab)** for isolated notebook environments

## Prerequisites

- Docker Desktop (or Docker Engine + Docker Compose plugin)
- ~2 GB of free disk space for the base images

## Quick start

```bash
# From the project root
docker-compose up --build
```

Once the stack is healthy you can reach the services at:

| Service         | URL                          | Notes |
|-----------------|------------------------------|-------|
| PHP application | http://localhost:8082        | Uses DB credentials from `docker-compose.yml` |
| phpMyAdmin      | http://localhost:8081        | Login with `dsp_user` / `dsp_pass` (or MySQL root) |
| JupyterHub      | https://localhost            | OAuth handshake redirects to your private notebook (published on port 443) |
| MySQL           | localhost:3307 (host access) | Database `niph_dsps`, user `dsp_user` / `dsp_pass` |

The first `docker-compose up` will import `db/niph_dsps.sql` automatically. Subsequent runs keep the data volume (`mysql_data`).

## Configuration

Key environment variables are defined in `docker-compose.yml`. Adjust them if you need different credentials or ports. The PHP application now reads its database configuration from the following variables (with sensible defaults for non-Docker setups):

- `DB_HOST`
- `DB_PORT`
- `DB_NAME`
- `DB_USER`
- `DB_PASS`

`api/run_r_script.php` also honours `RSCRIPT_PATH` if you need to override the default location of the `Rscript` executable.

When the portal is hosted on a different hostname (for example, an Ubuntu server on your LAN), set the following variables—either in your shell or a `.env` file consumed by Docker Compose—to keep the embedded JupyterHub session aligned with browser security rules:

- `JUPYTER_EXTERNAL_URL` – full base URL that the PHP app should point at (e.g. `https://niphdev.local`)
- `JUPYTERHUB_PORT` – published port if you map JupyterHub to something other than `443` (legacy deployments can continue to set `JUPYTER_PORT`)
- `DSP_APP_ORIGINS` – space-separated list of origins allowed to call notebook APIs (CORS)
- `DSP_FRAME_ANCESTORS` – space-separated list of origins permitted to embed JupyterHub in an iframe

### Platform roles at a glance

The application enforces the following roles via `ist_tbl_users.isu_status` and the helper functions in `includes/auth.php`. Use this matrix to confirm which actions (upload, read, download, approve) each role can take before issuing credentials:

| Role | Primary workspace | Upload / manage data sources | Approve access requests | Request / read / download datasets | Jupyter / R access |
|------|-------------------|------------------------------|-------------------------|------------------------------------|--------------------|
| **DAC Staff** | `admin/` area | ✅ Full oversight of every dataset, classification, and content entry. | ✅ Manage any permission, revoke and audit usage. | ✅ Can impersonate workflows when testing, but typically not used for research downloads. | ✅ Enable per-user via `isu_can_run_r`; also seeds OAuth credentials. |
| **Data Owner** | `data_owner/` | ✅ Create and maintain their own catalogue entries and metadata. | ✅ Approve, reject, or revoke requests for the data they own. | ✅ Access their own approved files plus anything they have requested from others. | ✅ Optional; grant by setting `isu_can_run_r = 1`. Only approved files sync into their notebook. |
| **Data Contributor** | `data_hybrid/` | ✅ Similar to owners, contributors can upload/publish datasets delegated to them. | ✅ Limited to the resources they registered or steward. | ✅ Can request access to other datasets and, once approved, read/download/analyze. | ✅ Optional per account; ideal for analysts who both publish and consume data. |
| **Data User** | `data_user/` | ❌ Browse-only catalogue view. | ❌ Cannot approve requests. | ✅ May request access, then read/download once a Data Owner or DAC Staff approves the request. | ✅ Optional; if enabled, only their approved files appear in Jupyter. |

> **Tip:** updating a user’s role or R access flag happens under **Admin → Manage Users**. Toggle the “Allow R/Jupyter” switch to control whether uploads are synchronized into their personal notebook volume.

To wire DSP into JupyterHub via OAuth, also provide:

- `DSP_OAUTH_CLIENT_ID` / `DSP_OAUTH_CLIENT_SECRET`
- `DSP_OAUTH_AUTHORIZE_URL`, `DSP_OAUTH_TOKEN_URL`, `DSP_OAUTH_USERINFO_URL`
- `JUPYTERHUB_OAUTH_CALLBACK`
- `JUPYTERHUB_USER_PATH` and `JUPYTERHUB_USERNAME_TEMPLATE` if you need custom routing/usernames
- `JUPYTERHUB_CULL_API_TOKEN` (optional) – set to enable the idle culler service

Seed or update the OAuth client after setting these env vars:

```bash
docker-compose exec app php scripts/seed_jupyterhub_client.php
```

The JupyterHub deployment trusts requests and iframe parents from `localhost:8082`, `127.0.0.1:8082`, and `https://dsp.niph.org.kh` by default. To allow different origins (for example your own DSP deployment), set:

- `DSP_APP_ORIGINS` – space-separated list of origins that should be accepted for CORS/websocket requests (e.g. `DSP_APP_ORIGINS="https://dsp.niph.org.kh"`).
- `DSP_FRAME_ANCESTORS` – space-separated list of origins allowed to embed the notebook in an iframe (e.g. `DSP_FRAME_ANCESTORS="https://dsp.niph.org.kh"`).

JupyterHub is published on host port `443` (configurable via the `JUPYTERHUB_PORT` environment variable in `docker-compose.yml`), so a deployment reachable at `https://dsp.niph.org.kh` works out of the box.

## Project directories shared with containers

| Host directory        | Container (app)         | Container (Jupyter)                |
|-----------------------|-------------------------|------------------------------------|
| `.` (project root)    | `/var/www/html`         | –                                  |
| `r_scripts/`          | `/var/www/html/r_scripts` | `/home/jovyan/work/r_scripts`      |
| `uploads/jupyter_workspace` | `/var/www/html/uploads/jupyter_workspace` | `/home/jovyan/work` (per-user mount inside spawned notebook) |

Uploads remain writable from the PHP container. If you run into permission warnings on macOS/Linux,
`chmod -R 777 uploads` (or a tighter group-based permission) on the host usually resolves it. The path is bind-mounted into the `dsp_app` container, so ensure permissions are adjusted on the host side.

- Uploaded files are stored under `uploads/datasources/` with names like `datasource_<unique>_<original-stem>.ext`. This keeps paths unique while preserving a readable hint of the original filename. The default PHP upload limit is set to `20M` (see `docker/custom.ini`).

- The `logs/app.log` file (created via `config.php`) records upload activity—if you do not see `[DataSource]` entries after an upload, confirm the app container can reach MySQL (`docker exec dsp_app php -r 'require "config.php"; echo "connected";'`).

## Architecture Overview

```mermaid
graph LR
    subgraph Client
        U[Browser / API Consumer]
    end

    subgraph Docker Stack
        A[PHP + Apache<br/>dsp_app]
        B[(MySQL 8.0<br/>dsp_db)]
        C[phpMyAdmin<br/>dsp_phpmyadmin]
        D[Jupyter Notebook<br/>dsp_jupyter]
        V1[(uploads/datasources)]
        V2[(r_scripts)]
    end

    U -->|HTTPS/HTTP :8082| A
    U -->|HTTPS/HTTP :8081| C
    U -->|HTTPS :443| D
    A <-->|SQL :3306| B
    C -->|Admin SQL| B
    A -.shared volume .-> V1
    A -.shared volume .-> V2
    D -.shared volume .-> V1
    D -.shared volume .-> V2
```

*Traffic legend:* solid lines represent runtime traffic, dotted lines represent bind-mounted volumes that synchronize datasets and R scripts between containers.

> Need the raw Mermaid for presentations? See `assets/diagrams/data_ecosystem.mmd`.

## Data Model Snapshot

```mermaid
erDiagram
    IST_TBL_PEOPLE ||--o{ IST_TBL_USERS : "fkisp_id_of"
    IST_TBL_PEOPLE ||--o{ DSPS_TBL_DATASOURCE : "fkisp_id_of"
    DSPS_TBL_TYPEDATASOURCE ||--o{ DSPS_TBL_DATASOURCE : "fkdspstds_id"
    DSPS_TBL_DSPSCATEGORY ||--o{ DSPS_TBL_DATASOURCE : "fkdspscate_id"
    DSPS_TBL_DATASOURCE ||--o{ DSPS_TBL_DATASOURCE_PERMISSION : "fkdspsds_id"
    IST_TBL_PEOPLE ||--o{ DSPS_TBL_DATASOURCE_PERMISSION : "fkisp_id_of (requester)"
    DSPS_TBL_DATASOURCE ||--o{ DSPS_TBL_DATASOURCE_USED : "fkdspsdsused_id"
    IST_TBL_PEOPLE ||--o{ DSPS_TBL_DATASOURCE_USED : "fkisp_id_of (consumer)"
```

The diagram highlights how every dataset anchors to a person record, while permissions and usage logs capture cross-person interactions for auditing.

## Analytics Catalog

Analytics scripts live in `r_scripts/` and are exposed through `api/run_r_script.php`. Each script receives two CLI arguments: the absolute path to a CSV prepared by PHP and a JSON string of runtime parameters.

| Script | Purpose | Required Parameters | Optional Parameters | Output |
|--------|---------|---------------------|---------------------|--------|
| `data_summary.R` | Smoke-test script that confirms connectivity between PHP and R, echoing the received file path and parameters. | _None_ | Any JSON payload is echoed back in `params_received`. | JSON with `message`, `data_file`, and the raw parameter string. |
| `descriptive_stats.R` | Generates descriptive statistics for every numeric column (count, mean, median, SD, min, max, missing) and returns up to five preview rows. | _None_ (operates on all numeric columns). | `encoding` (default `UTF-8`), `guess_max` to control type inference. | JSON payload containing `numeric_columns` keyed by column name plus `sample_rows`. Missing values are encoded as `null`. |
| `category_frequency.R` | Builds a frequency distribution for a categorical column. Useful for validating controlled vocabularies or spotting dominant categories. | `column` – name of the column to profile. | `top_n` (default `10`), `encoding` (default `UTF-8`), `include_missing` (`false` by default). | JSON with the analyzed column, configuration echo, and `frequencies` (value/count rows) sorted by frequency. |

### Adding another R script

1. Drop the script into `r_scripts/` and ensure it prints JSON via `jsonlite::toJSON(...)`.
2. Append the filename and human-readable label to `$allowed_r_scripts` inside `api/run_r_script.php`.
3. Document the new script in the table above so stakeholders understand its expected parameters and output contract.

## Useful commands

```bash
# Stop and remove containers, keeping the database volume
docker-compose down

# Stop containers and remove the database volume (fresh start)
docker-compose down -v

# Tail logs from all services
docker-compose logs -f
```

## Running Tests

PHPUnit is configured via Composer:

```bash
# Install dependencies (first run)
composer install

# Execute the test suite
composer test
```

If you prefer running inside the app container:

```bash
docker-compose exec app composer install
docker-compose exec app composer test
```

## Troubleshooting

- **MySQL already initialised**: remove the `mysql_data` named volume (`docker-compose down -v`) to force a clean import.
- **Rscript not found**: ensure the PHP container has R installed (`docker-compose build` again). Set `RSCRIPT_PATH` in `docker-compose.yml` if R lives elsewhere.
- **Port clashes**: adjust the published ports (`8082`, `8081`, `443`, `3307`) in `docker-compose.yml` to free ones on your machine.
- **Need the OAuth tables?**: run `docker-compose exec db mysql -u root -p niph_dsps < db/migrations/20241103_oauth_tables.sql` then insert your JupyterHub client credentials.

Happy hacking!