diff --git a/emqx/README-short.txt b/emqx/README-short.txt new file mode 100644 index 000000000000..86d9775bfe12 --- /dev/null +++ b/emqx/README-short.txt @@ -0,0 +1 @@ +An Open-Source, Cloud-Native, Distributed MQTT Message Broker for IoT. diff --git a/emqx/content.md b/emqx/content.md new file mode 100644 index 000000000000..61945ad6d25f --- /dev/null +++ b/emqx/content.md @@ -0,0 +1,305 @@ +# What is EMQ X ? + +[EMQ X MQTT broker](https://www.emqx.io/) is a fully open source, highly scalable, highly available distributed MQTT messaging broker for IoT, M2M and Mobile applications that can handle tens of millions of concurrent clients. + +Starting from 3.0 release, *EMQ X* broker fully supports MQTT V5.0 protocol specifications and backward compatible with MQTT V3.1 and V3.1.1, as well as other communication protocols such as MQTT-SN, CoAP, LwM2M, WebSocket and STOMP. The 3.0 release of the *EMQ X* broker can scaled to 10+ million concurrent MQTT connections on one cluster. + +%%LOGO%% + +# How to use this image + +### Run emqx + +Execute some command under this docker image + +```console +$ docker run -d --name emqx %%IMAGE%%:tag +``` + +For example + +```console +$ docker run -d --name emqx -p 18083:18083 -p 1883:1883 %%IMAGE%%:latest +``` + +The emqx broker runs as linux user `emqx` in the docker container. + +### Configuration + +Use the environment variable to configure the EMQ X docker container. + +By default, the environment variables with `EMQX_` prefix are mapped to key-value pairs in configuration files. + +You can change the prefix by overriding `HOCON_ENV_OVERRIDE_PREFIX`. + +Example: + +```bash +EMQX_LISTENERS__SSL__DEFAULT__ACCEPTORS <--> listeners.ssl.default.acceptors +EMQX_ZONES__DEFAULT__MQTT__MAX_PACKET_SIZE <--> zones.default.mqtt.max_packet_size +``` + +- Prefix `EMQX_` is removed +- All upper case letters is replaced with lower case letters +- `__` is replaced with `.` + +If `HOCON_ENV_OVERRIDE_PREFIX=DEV_` is set: + +```bash +DEV_LISTENER__SSL__EXTERNAL__ACCEPTORS <--> listener.ssl.external.acceptors +DEV_MQTT__MAX_PACKET_SIZE <--> mqtt.max_packet_size +``` + +Non mapped environment variables: + +```bash +EMQX_NAME +EMQX_HOST +``` + +These environment variables will ignore for configuration file. + +#### EMQ X Configuration + +> NOTE: All EMQ X Configuration in [etc/emqx.conf](https://github.com/emqx/emqx/blob/master/etc/emqx.conf) could config by environment. The following list is just an example, not a complete configuration. + +| Options | Default | Mapped | Description | +|-------------|----------------|--------|----------------------------| +| `EMQX_NAME` | container name | none | emqx node short name | +| `EMQX_HOST` | container IP | none | emqx node host, IP or FQDN | + +The list is incomplete and may changed with [etc/emqx.conf](https://github.com/emqx/emqx/blob/master/etc/emqx.conf) and plugin configuration files. But the mapping rule is similar. + +If set `EMQX_NAME` and `EMQX_HOST`, and unset `EMQX_NODE_NAME`, `EMQX_NODE_NAME=$EMQX_NAME@$EMQX_HOST`. + +For example, set mqtt tcp port to 1883 + +```console +$ docker run -d --name emqx -e EMQX__LISTENERS__TCP__DEFAULT__BIND=1883 -p 18083:18083 -p 1883:1883 %%IMAGE%%:latest +``` + +#### EMQ Loaded Modules Configuration + +| Options | Default | Description | +|-----------------------|-------------------|-----------------------------| +| `EMQX_LOADED_MODULES` | see content below | default modules emqx loaded | + +Default environment variable `EMQX_LOADED_MODULES`, including + +- `emqx_mod_presence` + +```bash +# The default EMQX_LOADED_MODULES env +EMQX_LOADED_MODULES="emqx_mod_presence" +``` + +For example, set `EMQX_LOADED_MODULES=emqx_mod_delayed,emqx_mod_rewrite` to load these two modules. + +You can use comma, space or other separator that you want. + +All the modules defined in env `EMQX_LOADED_MODULES` will be loaded. + +```bash +EMQX_LOADED_MODULES="emqx_mod_delayed,emqx_mod_rewrite" +EMQX_LOADED_MODULES="emqx_mod_delayed emqx_mod_rewrite" +EMQX_LOADED_MODULES="emqx_mod_delayed | emqx_mod_rewrite" +``` + +#### EMQ Loaded Plugins Configuration + +| Options | Default | Description | +|-----------------------|-------------------|-----------------------------| +| `EMQX_LOADED_PLUGINS` | see content below | default plugins emqx loaded | + +Default environment variable `EMQX_LOADED_PLUGINS`, including + +- `emqx_recon` +- `emqx_retainer` +- `emqx_rule_engine` +- `emqx_management` +- `emqx_dashboard` + +```bash +# The default EMQX_LOADED_PLUGINS env +EMQX_LOADED_PLUGINS="emqx_recon,emqx_retainer,emqx_management,emqx_dashboard" +``` + +For example, set `EMQX_LOADED_PLUGINS= emqx_retainer,emqx_rule_engine` to load these two plugins. + +You can use comma, space or other separator that you want. + +All the plugins defined in `EMQX_LOADED_PLUGINS` will be loaded. + +```bash +EMQX_LOADED_PLUGINS="emqx_retainer,emqx_rule_engine" +EMQX_LOADED_PLUGINS="emqx_retainer emqx_rule_engine" +EMQX_LOADED_PLUGINS="emqx_retainer | emqx_rule_engine" +``` + +#### EMQ X Plugins Configuration + +The environment variables which with `EMQX_` prefix are mapped to all emqx plugins' configuration file, `.` get replaced by `__`. + +Example: + +```bash +EMQX_RETAINER__STORAGE_TYPE <--> retainer.storage_type +EMQX_RETAINER__MAX_PAYLOAD_SIZE <--> retainer.max_payload_size +``` + +Don't worry about where to find the configuration file of emqx plugins, this docker image will find and config them automatically using some magic. + +All plugin of emqx project could config in this way, following the environment variables mapping rule above. + +Assume you are using redis auth plugin, for example: + +```bash +#EMQX_RETAINER__STORAGE_TYPE = "ram" +#EMQX_RETAINER.MAX_PAYLOAD_SIZE = 1MB + +docker run -d --name emqx -p 18083:18083 -p 1883:1883 -p 4369:4369 \ + -e EMQX_LISTENERS__TCP__DEFAULT=1883 \ + -e EMQX_LOADED_PLUGINS="emqx_retainer" \ + -e EMQX_RETAINER__STORAGE_TYPE = "ram" \ + -e EMQX_RETAINER__MAX_PAYLOAD_SIZE = 1MB \ + %%IMAGE%%:latest +``` + +For numbered configuration options where the number is next to a `.` such as: + +- backend.redis.pool1.server +- backend.redis.hook.message.publish.1 + +You can configure an arbitrary number of them as long as each has a uniq unber for it's own configuration option: + +```bash +docker run -d --name emqx -p 18083:18083 -p 1883:1883 -p 4369:4369 \ + -e EMQX_BACKEND_REDIS_POOL1__SERVER=127.0.0.1:6379 \ + [...] + -e EMQX_BACKEND__REDIS__POOL5__SERVER=127.0.0.5:6379 \ + -e EMQX_BACKEND__REDIS__HOOK_MESSAGE__PUBLISH__1='{"topic": "persistant/topic1", "action": {"function": "on_message_publish"}, "pool": "pool1"}' \ + -e EMQX_BACKEND__REDIS__HOOK_MESSAGE__PUBLISH__2='{"topic": "persistant/topic2", "action": {"function": "on_message_publish"}, "pool": "pool1"}' \ + -e EMQX_BACKEND__REDIS__HOOK_MESSAGE__PUBLISH__3='{"topic": "persistant/topic3", "action": {"function": "on_message_publish"}, "pool": "pool1"}' \ + [...] + -e EMQX_BACKEND__REDIS__HOOK_MESSAGE__PUBLISH__13='{"topic": "persistant/topic13", "action": {"function": "on_message_publish"}, "pool": "pool1"}' \ + %%IMAGE%%:latest +``` + +### Cluster + +EMQ X supports a variety of clustering methods, see our [documentation](https://docs.emqx.io/broker/latest/en/advanced/cluster.html#emqx-service-discovery) for details. + +Let's create a static node list cluster from docker-compose. + +- Create `docker-compose.yaml`: + +```yaml + version: '3' + + services: + emqx1: + image: %%IMAGE%%:latest + environment: + - "EMQX_NAME=emqx" + - "EMQX_HOST=node1.emqx.io" + - "EMQX_CLUSTER__DISCOVERY_STRATEGY=static" + - "EMQX_CLUSTER__STATIC__SEEDS=[emqx@node1.emqx.io, emqx@node2.emqx.io]" + networks: + emqx-bridge: + aliases: + - node1.emqx.io + + emqx2: + image: %%IMAGE%%:latest + environment: + - "EMQX_NAME=emqx" + - "EMQX_HOST=node2.emqx.io" + - "EMQX_CLUSTER__DISCOVERY_STRATEGY=static" + - "EMQX_CLUSTER__STATIC__SEEDS=[emqx@node1.emqx.io, emqx@node2.emqx.io]" + networks: + emqx-bridge: + aliases: + - node2.emqx.io + + networks: + emqx-bridge: + driver: bridge +``` + +- Start the docker-compose cluster + +```bash + docker-compose -p my_emqx up -d +``` + +- View cluster + +```bash + $ docker exec -it my_emqx_emqx1_1 sh -c "emqx_ctl cluster status" + Cluster status: #{running_nodes => ['emqx@node1.emqx.io','emqx@node2.emqx.io'], + stopped_nodes => []} +``` + +### Persistence + +If you want to persist the EMQ X docker container, you need to keep the following directories: + +- `/opt/emqx/data` +- `/opt/emqx/etc` +- `/opt/emqx/log` + +Since data in these folders are partially stored under the `/opt/emqx/data/mnesia/${node_name}`, the user also needs to reuse the same node name to see the previous state. In detail, one needs to specify the two environment variables: `EMQX_NAME` and `EMQX_HOST`, `EMQX_HOST` set as `127.0.0.1` or network alias would be useful. + +In if you use docker-compose, the configuration would look something like this: + +```YAML +volumes: + vol-emqx-data: + name: foo-emqx-data + vol-emqx-etc: + name: foo-emqx-etc + vol-emqx-log: + name: foo-emqx-log + +services: + emqx: + image: %%IMAGE%%:latest + restart: always + environment: + EMQX_NAME: foo_emqx + EMQX_HOST: 127.0.0.1 + volumes: + - vol-emqx-data:/opt/emqx/data + + - vol-emqx-log:/opt/emqx/log +``` + +### Kernel Tuning + +Under linux host machine, the easiest way is [Tuning guide](https://docs.emqx.io/en/broker/latest/tutorial/tune.html#linux-kernel-tuning). + +If you want tune linux kernel by docker, you must ensure your docker is latest version (>=1.12). + +```bash + +docker run -d --name emqx -p 18083:18083 -p 1883:1883 -p 4369:4369 \ + --sysctl fs.file-max=2097152 \ + --sysctl fs.nr_open=2097152 \ + --sysctl net.core.somaxconn=32768 \ + --sysctl net.ipv4.tcp_max_syn_backlog=16384 \ + --sysctl net.core.netdev_max_backlog=16384 \ + --sysctl net.ipv4.ip_local_port_range=1000 65535 \ + --sysctl net.core.rmem_default=262144 \ + --sysctl net.core.wmem_default=262144 \ + --sysctl net.core.rmem_max=16777216 \ + --sysctl net.core.wmem_max=16777216 \ + --sysctl net.core.optmem_max=16777216 \ + --sysctl net.ipv4.tcp_rmem=1024 4096 16777216 \ + --sysctl net.ipv4.tcp_wmem=1024 4096 16777216 \ + --sysctl net.ipv4.tcp_max_tw_buckets=1048576 \ + --sysctl net.ipv4.tcp_fin_timeout=15 \ + %%IMAGE%%:latest + +``` + +> REMEMBER: DO NOT RUN EMQ X DOCKER PRIVILEGED OR MOUNT SYSTEM PROC IN CONTAINER TO TUNE LINUX KERNEL, IT IS UNSAFE. diff --git a/emqx/get-help.md b/emqx/get-help.md new file mode 100644 index 000000000000..75c73bfa32c7 --- /dev/null +++ b/emqx/get-help.md @@ -0,0 +1 @@ +[Discussions](https://github.com/emqx/emqx/discussions) or [Slack](https://slack-invite.emqx.io/) diff --git a/emqx/github-repo b/emqx/github-repo new file mode 100644 index 000000000000..c225c8ed2658 --- /dev/null +++ b/emqx/github-repo @@ -0,0 +1 @@ +https://github.com/emqx/emqx-docker diff --git a/emqx/license.md b/emqx/license.md new file mode 100644 index 000000000000..e2a94a5f02ba --- /dev/null +++ b/emqx/license.md @@ -0,0 +1 @@ +View [license information](https://github.com/emqx/emqx/blob/master/LICENSE) for the software contained in this image. diff --git a/emqx/logo.png b/emqx/logo.png new file mode 100644 index 000000000000..799176a7e0fa Binary files /dev/null and b/emqx/logo.png differ diff --git a/emqx/maintainer.md b/emqx/maintainer.md new file mode 100644 index 000000000000..3faa249415e0 --- /dev/null +++ b/emqx/maintainer.md @@ -0,0 +1 @@ +[EMQ Technologies](https://github.com/emqx)