From 6a4c1bf7ccbc2df9f3447a2d86f162727d4595be Mon Sep 17 00:00:00 2001 From: Mehmet TOSUN Date: Sun, 31 Aug 2025 06:42:28 +0300 Subject: [PATCH 1/9] Add ClickHouse state store documentation - Add comprehensive documentation for ClickHouse state store component - Include setup instructions for self-hosted, Kubernetes, and cloud deployments - Document supported features: CRUD, ETag, and TTL support - Add advanced configuration options and performance considerations - Update component registry with ClickHouse entry in generic.yaml - Set correct version as 1.16 (first release) - Remove actor-related content as ClickHouse is not transactional Signed-off-by: Mehmet TOSUN --- .../setup-clickhouse.md | 193 ++++++++++++++++++ .../data/components/state_stores/generic.yaml | 11 + 2 files changed, 204 insertions(+) create mode 100644 daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md new file mode 100644 index 00000000000..2570f4831b3 --- /dev/null +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md @@ -0,0 +1,193 @@ +--- +type: docs +title: "ClickHouse" +linkTitle: "ClickHouse" +description: Detailed information on the ClickHouse state store component +aliases: + - "/operations/components/setup-state-store/supported-state-stores/setup-clickhouse/" +--- + +## Component format + +To setup ClickHouse state store create a component of type `state.clickhouse`. See [this guide]({{< ref "howto-get-save-state.md#step-1-setup-a-state-store" >}}) on how to create and apply a state store configuration. + +```yaml +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: +spec: + type: state.clickhouse + version: v1 + metadata: + - name: clickhouseURL + value: + - name: databaseName + value: + - name: tableName + value: + - name: username # Optional + value: + - name: password # Optional + value: +``` + +{{% alert title="Warning" color="warning" %}} +The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}). +{{% /alert %}} + +## Spec metadata fields + +| Field | Required | Details | Example | +|--------------------|:--------:|---------|---------| +| clickhouseURL | Y | Connection URL for the ClickHouse server | `"clickhouse://localhost:9000"`, `"clickhouse://clickhouse-server:9000"` | +| databaseName | Y | Name of the database to use | `"dapr_state"`, `"my_database"` | +| tableName | Y | Name of the table to store state data | `"state_table"`, `"dapr_state_store"` | +| username | N | Username for ClickHouse authentication. Can be `secretKeyRef` to use a secret reference | `"default"`, `"my_user"` | +| password | N | Password for ClickHouse authentication. Can be `secretKeyRef` to use a secret reference | `"my_password"` | + +## Setup ClickHouse + +Dapr can use any ClickHouse instance: containerized, running on your local dev machine, or a managed cloud service. + +{{< tabs "Self-Hosted" "Kubernetes" "Cloud" >}} + +{{% codetab %}} + +1. Run an instance of ClickHouse. You can run a local instance of ClickHouse in Docker with the following command: + + ```bash + docker run -d --name clickhouse-server \ + -p 8123:8123 -p 9000:9000 \ + -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 \ + -e CLICKHOUSE_PASSWORD=my_password \ + clickhouse/clickhouse-server + ``` + +2. Create a database for state data (optional, as Dapr will create it automatically): + + ```sql + CREATE DATABASE IF NOT EXISTS dapr_state; + ``` + +{{% /codetab %}} + +{{% codetab %}} + +You can use [Helm](https://helm.sh/) to quickly create a ClickHouse instance in your Kubernetes cluster. This approach requires [Installing Helm](https://github.com/helm/helm#install). + +1. Add the ClickHouse Helm repository: + ```bash + helm repo add clickhouse https://docs.altinity.com/clickhouse-operator/ + helm repo update + ``` + +2. Install ClickHouse into your cluster: + ```bash + helm install clickhouse clickhouse/clickhouse + ``` + +3. Run `kubectl get pods` to see the ClickHouse containers now running in your cluster. + +4. Add the ClickHouse service endpoint as the `clickhouseURL` in your component configuration. For example: + ```yaml + metadata: + - name: clickhouseURL + value: "clickhouse://clickhouse:9000" + ``` + +{{% /codetab %}} + +{{% codetab %}} + +ClickHouse is available as a managed service from various cloud providers: + +- [ClickHouse Cloud](https://clickhouse.com/cloud) +- [Altinity.Cloud](https://altinity.com/cloud-database/) +- [Yandex Managed Service for ClickHouse](https://cloud.yandex.com/services/managed-clickhouse) + +When using a managed service, ensure you have the correct connection URL, database name, and credentials configured in your component metadata. + +{{% /codetab %}} + +{{< /tabs >}} + +## Features + +The ClickHouse state store supports the following features: + +### ETags + +The ClickHouse state store supports [ETags]({{< ref state-management-overview.md >}}) for optimistic concurrency control. ETags are automatically generated and updated when state data is modified. + +### TTL (Time-To-Live) + +This state store supports [Time-To-Live (TTL)]({{< ref state-store-ttl.md >}}) for records stored with Dapr. When storing data using Dapr, you can set the `ttlInSeconds` metadata property to indicate after how many seconds the data should be considered "expired". + +Example of setting TTL: + +```json +{ + "key": "my-key", + "value": "my-value", + "metadata": { + "ttlInSeconds": "3600" + } +} +``` + +Records with expired TTLs are automatically filtered out during read operations and are eligible for cleanup by ClickHouse's background processes. + +## Advanced + +### Table Schema + +The ClickHouse state store creates a table with the following schema: + +```sql +CREATE TABLE IF NOT EXISTS . ( + key String, + value String, + etag String, + expire DateTime64(3) NULL, + PRIMARY KEY(key) +) ENGINE = ReplacingMergeTree() +ORDER BY key +``` + +The table uses ClickHouse's `ReplacingMergeTree` engine, which automatically deduplicates rows with the same primary key during background merges. + +### Connection URL Format + +The ClickHouse connection URL follows the standard format: + +``` +clickhouse://[username[:password]@]host[:port][/database][?param1=value1&...¶mN=valueN] +``` + +Examples: +- `clickhouse://localhost:9000` +- `clickhouse://user:password@clickhouse-server:9000/my_db` +- `clickhouse://localhost:9000?dial_timeout=10s&max_execution_time=60` + +### Performance Considerations + +- The ClickHouse state store is optimized for high-throughput scenarios +- For better performance with large datasets, consider partitioning your table by date or other relevant columns +- The `ReplacingMergeTree` engine provides eventual consistency for duplicate key handling +- Background merges in ClickHouse will automatically clean up old versions of updated records + +### Bulk Operations + +The ClickHouse state store supports bulk operations for improved performance: + +- `BulkGet`: Retrieve multiple keys in a single operation +- `BulkSet`: Store multiple key-value pairs in a single operation +- `BulkDelete`: Delete multiple keys in a single operation + +## Related links + +- [Basic schema for a Dapr component]({{< ref component-schema >}}) +- Read [this guide]({{< ref "howto-get-save-state.md#step-2-save-and-retrieve-a-single-state" >}}) for instructions on configuring state store components +- [State management building block]({{< ref state-management >}}) +- [ClickHouse Official Documentation](https://clickhouse.com/docs) \ No newline at end of file diff --git a/daprdocs/data/components/state_stores/generic.yaml b/daprdocs/data/components/state_stores/generic.yaml index ee5ca782c2f..dbb61320eae 100644 --- a/daprdocs/data/components/state_stores/generic.yaml +++ b/daprdocs/data/components/state_stores/generic.yaml @@ -20,6 +20,17 @@ etag: false ttl: true query: false +- component: ClickHouse + link: setup-clickhouse + state: Alpha + version: v1 + since: "1.16" + features: + crud: true + transactions: false + etag: true + ttl: true + query: false - component: CockroachDB link: setup-cockroachdb state: Stable From b21e0c2f0cc9df79a31ecf80d73c1ffed694c95d Mon Sep 17 00:00:00 2001 From: Marc Duiker Date: Thu, 11 Sep 2025 11:32:06 +0200 Subject: [PATCH 2/9] Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md Signed-off-by: Marc Duiker --- .../supported-state-stores/setup-clickhouse.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md index 2570f4831b3..d07d9092c90 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md @@ -50,7 +50,7 @@ The above example uses secrets as plain strings. It is recommended to use a secr Dapr can use any ClickHouse instance: containerized, running on your local dev machine, or a managed cloud service. -{{< tabs "Self-Hosted" "Kubernetes" "Cloud" >}} +{{< tabpane text=true >}} {{% codetab %}} From 3f139c3bd1d89782890fc2bbd110970b729bd036 Mon Sep 17 00:00:00 2001 From: Marc Duiker Date: Thu, 11 Sep 2025 11:32:15 +0200 Subject: [PATCH 3/9] Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md Signed-off-by: Marc Duiker --- .../supported-state-stores/setup-clickhouse.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md index d07d9092c90..71c7622862c 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md @@ -52,7 +52,7 @@ Dapr can use any ClickHouse instance: containerized, running on your local dev m {{< tabpane text=true >}} -{{% codetab %}} +{{% tab "Self-Hosted" %}} 1. Run an instance of ClickHouse. You can run a local instance of ClickHouse in Docker with the following command: From 4981bd3d7ff57858c6d3d977ccc7ee0319f0d99e Mon Sep 17 00:00:00 2001 From: Marc Duiker Date: Thu, 11 Sep 2025 11:32:24 +0200 Subject: [PATCH 4/9] Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md Signed-off-by: Marc Duiker --- .../supported-state-stores/setup-clickhouse.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md index 71c7622862c..b3af28a6483 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md @@ -70,7 +70,7 @@ Dapr can use any ClickHouse instance: containerized, running on your local dev m CREATE DATABASE IF NOT EXISTS dapr_state; ``` -{{% /codetab %}} +{{% /tab %}} {{% codetab %}} From 1f03d4027be182df18e835d325050256db9a3101 Mon Sep 17 00:00:00 2001 From: Marc Duiker Date: Thu, 11 Sep 2025 11:32:32 +0200 Subject: [PATCH 5/9] Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md Signed-off-by: Marc Duiker --- .../supported-state-stores/setup-clickhouse.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md index b3af28a6483..7feff6f6e05 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md @@ -72,7 +72,7 @@ Dapr can use any ClickHouse instance: containerized, running on your local dev m {{% /tab %}} -{{% codetab %}} +{{% tab "Kubernetes" %}} You can use [Helm](https://helm.sh/) to quickly create a ClickHouse instance in your Kubernetes cluster. This approach requires [Installing Helm](https://github.com/helm/helm#install). From 0e706ade3fcb3c99a530a162ee935cf58a58379d Mon Sep 17 00:00:00 2001 From: Marc Duiker Date: Thu, 11 Sep 2025 11:32:40 +0200 Subject: [PATCH 6/9] Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md Signed-off-by: Marc Duiker --- .../supported-state-stores/setup-clickhouse.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md index 7feff6f6e05..2d5a13cc3e4 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md @@ -96,7 +96,7 @@ You can use [Helm](https://helm.sh/) to quickly create a ClickHouse instance in value: "clickhouse://clickhouse:9000" ``` -{{% /codetab %}} +{{% /tab %}} {{% codetab %}} From 730faebd70f97e3fc9e3fbfc1f2d7df425985fbd Mon Sep 17 00:00:00 2001 From: Marc Duiker Date: Thu, 11 Sep 2025 11:32:47 +0200 Subject: [PATCH 7/9] Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md Signed-off-by: Marc Duiker --- .../supported-state-stores/setup-clickhouse.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md index 2d5a13cc3e4..e7544715408 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md @@ -98,7 +98,7 @@ You can use [Helm](https://helm.sh/) to quickly create a ClickHouse instance in {{% /tab %}} -{{% codetab %}} +{{% tab "Cloud" %}} ClickHouse is available as a managed service from various cloud providers: From 1c9b297db413b8737594d47b5673eac81abf46f4 Mon Sep 17 00:00:00 2001 From: Marc Duiker Date: Thu, 11 Sep 2025 11:32:55 +0200 Subject: [PATCH 8/9] Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md Signed-off-by: Marc Duiker --- .../supported-state-stores/setup-clickhouse.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md index e7544715408..a6e69ae8326 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md @@ -108,7 +108,7 @@ ClickHouse is available as a managed service from various cloud providers: When using a managed service, ensure you have the correct connection URL, database name, and credentials configured in your component metadata. -{{% /codetab %}} +{{% /tab %}} {{< /tabs >}} From ecd3815b093122b421cd3cc388b8faab22d6d339 Mon Sep 17 00:00:00 2001 From: Marc Duiker Date: Thu, 11 Sep 2025 11:33:02 +0200 Subject: [PATCH 9/9] Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md Signed-off-by: Marc Duiker --- .../supported-state-stores/setup-clickhouse.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md index a6e69ae8326..23e7893ad4d 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md @@ -110,7 +110,7 @@ When using a managed service, ensure you have the correct connection URL, databa {{% /tab %}} -{{< /tabs >}} +{{< /tabpane >}} ## Features