chore: Update dev setup scripts and API README (#31415)

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

api/README.md

@@ -1,6 +1,6 @@
 # Dify Backend API
 
-## Usage
+## Setup and Run
 
 > [!IMPORTANT]
 >
@@ -8,48 +8,77 @@
 > [`uv`](https://docs.astral.sh/uv/) as the package manager
 > for Dify API backend service.
 
-1. Start the docker-compose stack
+`uv` and `pnpm` are required to run the setup and development commands below.
 
-   The backend require some middleware, including PostgreSQL, Redis, and Weaviate, which can be started together using `docker-compose`.
+### Using scripts (recommended)
+
+The scripts resolve paths relative to their location, so you can run them from anywhere.
+
+1. Run setup (copies env files and installs dependencies).
 
    ```bash
-   cd ../docker
+   ./dev/setup
-   cp middleware.env.example middleware.env
-   # change the profile to mysql if you are not using postgres,change the profile to other vector database if you are not using weaviate
-   docker compose -f docker-compose.middleware.yaml --profile postgresql --profile weaviate -p dify up -d
-   cd ../api
    ```
 
-1. Copy `.env.example` to `.env`
+1. Review `api/.env`, `web/.env.local`, and `docker/middleware.env` values (see the `SECRET_KEY` note below).
 
-   ```cli
+1. Start middleware (PostgreSQL/Redis/Weaviate).
-   cp .env.example .env
+
+   ```bash
+   ./dev/start-docker-compose
    ```
 
-> [!IMPORTANT]
+1. Start backend (runs migrations first).
->
-> When the frontend and backend run on different subdomains, set COOKIE_DOMAIN to the site’s top-level domain (e.g., `example.com`). The frontend and backend must be under the same top-level domain in order to share authentication cookies.
 
-1. Generate a `SECRET_KEY` in the `.env` file.
+   ```bash
-
+   ./dev/start-api
-   bash for Linux
-
-   ```bash for Linux
-   sed -i "/^SECRET_KEY=/c\SECRET_KEY=$(openssl rand -base64 42)" .env
    ```
 
-   bash for Mac
+1. Start Dify [web](../web) service.
 
-   ```bash for Mac
+   ```bash
-   secret_key=$(openssl rand -base64 42)
+   ./dev/start-web
-   sed -i '' "/^SECRET_KEY=/c\\
-   SECRET_KEY=${secret_key}" .env
    ```
 
-1. Create environment.
+1. Set up your application by visiting `http://localhost:3000`.
 
-   Dify API service uses [UV](https://docs.astral.sh/uv/) to manage dependencies.
+1. Optional: start the worker service (async tasks, runs from `api`).
-   First, you need to add the uv package manager, if you don't have it already.
+
+   ```bash
+   ./dev/start-worker
+   ```
+
+1. Optional: start Celery Beat (scheduled tasks).
+
+   ```bash
+   ./dev/start-beat
+   ```
+
+### Manual commands
+
+<details>
+<summary>Show manual setup and run steps</summary>
+
+These commands assume you start from the repository root.
+
+1. Start the docker-compose stack.
+
+   The backend requires middleware, including PostgreSQL, Redis, and Weaviate, which can be started together using `docker-compose`.
+
+   ```bash
+   cp docker/middleware.env.example docker/middleware.env
+   # Use mysql or another vector database profile if you are not using postgres/weaviate.
+   docker compose -f docker/docker-compose.middleware.yaml --profile postgresql --profile weaviate -p dify up -d
+   ```
+
+1. Copy env files.
+
+   ```bash
+   cp api/.env.example api/.env
+   cp web/.env.example web/.env.local
+   ```
+
+1. Install UV if needed.
 
    ```bash
    pip install uv
@@ -57,60 +86,96 @@
    brew install uv
    ```
 
-1. Install dependencies
+1. Install API dependencies.
 
    ```bash
-   uv sync --dev
+   cd api
+   uv sync --group dev
    ```
 
-1. Run migrate
+1. Install web dependencies.
-
-   Before the first launch, migrate the database to the latest version.
 
    ```bash
+   cd web
+   pnpm install
+   cd ..
+   ```
+
+1. Start backend (runs migrations first, in a new terminal).
+
+   ```bash
+   cd api
    uv run flask db upgrade
-   ```
-
-1. Start backend
-
-   ```bash
    uv run flask run --host 0.0.0.0 --port=5001 --debug
    ```
 
-1. Start Dify [web](../web) service.
+1. Start Dify [web](../web) service (in a new terminal).
 
-1. Setup your application by visiting `http://localhost:3000`.
+   ```bash
+   cd web
+   pnpm dev:inspect
+   ```
 
-1. If you need to handle and debug the async tasks (e.g. dataset importing and documents indexing), please start the worker service.
+1. Set up your application by visiting `http://localhost:3000`.
 
-```bash
+1. Optional: start the worker service (async tasks, in a new terminal).
-uv run celery -A app.celery worker -P threads -c 2 --loglevel INFO -Q dataset,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor,retention
-```
 
-Additionally, if you want to debug the celery scheduled tasks, you can run the following command in another terminal to start the beat service:
+   ```bash
+   cd api
+   uv run celery -A app.celery worker -P threads -c 2 --loglevel INFO -Q dataset,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor,retention
+   ```
 
-```bash
+1. Optional: start Celery Beat (scheduled tasks, in a new terminal).
-uv run celery -A app.celery beat
+
-```
+   ```bash
+   cd api
+   uv run celery -A app.celery beat
+   ```
+
+</details>
+
+### Environment notes
+
+> [!IMPORTANT]
+>
+> When the frontend and backend run on different subdomains, set COOKIE_DOMAIN to the site’s top-level domain (e.g., `example.com`). The frontend and backend must be under the same top-level domain in order to share authentication cookies.
+
+- Generate a `SECRET_KEY` in the `.env` file.
+
+  bash for Linux
+
+  ```bash
+  sed -i "/^SECRET_KEY=/c\\SECRET_KEY=$(openssl rand -base64 42)" .env
+  ```
+
+  bash for Mac
+
+  ```bash
+  secret_key=$(openssl rand -base64 42)
+  sed -i '' "/^SECRET_KEY=/c\\
+  SECRET_KEY=${secret_key}" .env
+  ```
 
 ## Testing
 
 1. Install dependencies for both the backend and the test environment
 
    ```bash
-   uv sync --dev
+   cd api
+   uv sync --group dev
    ```
 
 1. Run the tests locally with mocked system environment variables in `tool.pytest_env` section in `pyproject.toml`, more can check [Claude.md](../CLAUDE.md)
 
    ```bash
+   cd api
    uv run pytest                           # Run all tests
    uv run pytest tests/unit_tests/         # Unit tests only
    uv run pytest tests/integration_tests/  # Integration tests
 
    # Code quality
-   ../dev/reformat               # Run all formatters and linters
+   ./dev/reformat               # Run all formatters and linters
-   uv run ruff check --fix ./    # Fix linting issues
+   uv run ruff check --fix ./   # Fix linting issues
-   uv run ruff format ./         # Format code
+   uv run ruff format ./        # Format code
-   uv run basedpyright .         # Type checking
+   uv run basedpyright .        # Type checking
    ```

dev/setup

@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(dirname "$(realpath "$0")")"
+ROOT="$(dirname "$SCRIPT_DIR")"
+
+API_ENV_EXAMPLE="$ROOT/api/.env.example"
+API_ENV="$ROOT/api/.env"
+WEB_ENV_EXAMPLE="$ROOT/web/.env.example"
+WEB_ENV="$ROOT/web/.env.local"
+MIDDLEWARE_ENV_EXAMPLE="$ROOT/docker/middleware.env.example"
+MIDDLEWARE_ENV="$ROOT/docker/middleware.env"
+
+# 1) Copy api/.env.example -> api/.env
+cp "$API_ENV_EXAMPLE" "$API_ENV"
+
+# 2) Copy web/.env.example -> web/.env.local
+cp "$WEB_ENV_EXAMPLE" "$WEB_ENV"
+
+# 3) Copy docker/middleware.env.example -> docker/middleware.env
+cp "$MIDDLEWARE_ENV_EXAMPLE" "$MIDDLEWARE_ENV"
+
+# 4) Install deps
+cd "$ROOT/api"
+uv sync --group dev
+
+cd "$ROOT/web"
+pnpm install

dev/start-api

@@ -3,8 +3,9 @@
 set -x
 
 SCRIPT_DIR="$(dirname "$(realpath "$0")")"
-cd "$SCRIPT_DIR/.."
+cd "$SCRIPT_DIR/../api"
 
+uv run flask db upgrade
 
-uv --directory api run \
+uv run \
   flask run --host 0.0.0.0 --port=5001 --debug

dev/start-docker-compose

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(dirname "$(realpath "$0")")"
+ROOT="$(dirname "$SCRIPT_DIR")"
+
+cd "$ROOT/docker"
+docker compose -f docker-compose.middleware.yaml --profile postgresql --profile weaviate -p dify up -d

dev/start-worker

@@ -83,7 +83,7 @@ while [[ $# -gt 0 ]]; do
 done
 
 SCRIPT_DIR="$(dirname "$(realpath "$0")")"
-cd "$SCRIPT_DIR/.."
+cd "$SCRIPT_DIR/../api"
 
 if [[ -n "${ENV_FILE}" ]]; then
   if [[ ! -f "${ENV_FILE}" ]]; then
@@ -123,6 +123,6 @@ echo "  Concurrency: ${CONCURRENCY}"
 echo "  Pool: ${POOL}"
 echo "  Log Level: ${LOGLEVEL}"
 
-uv --directory api run \
+uv run \
   celery -A app.celery worker \
   -P ${POOL} -c ${CONCURRENCY} --loglevel ${LOGLEVEL} -Q ${QUEUES}

dev/update-uv

@@ -4,7 +4,7 @@
 set -e
 set -o pipefail
 
-SCRIPT_DIR="$(dirname "$0")"
+SCRIPT_DIR="$(dirname "$(realpath "$0")")"
 REPO_ROOT="$(dirname "${SCRIPT_DIR}")"
 
 # rely on `poetry` in path