MQTT¶

Overview¶

MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe messaging protocol designed for IoT devices and low-bandwidth, high-latency, or unreliable networks. It follows a client-server architecture with a central broker handling message routing.

Data Model¶

Core Concepts¶

graph TB
    subgraph "MQTT Broker"
        BROKER[MQTT Broker]
        TOPICS[Topic Tree]
        SUBS[Subscriptions]
        RETAINED[Retained Messages]
    end

    subgraph "MQTT Clients"
        PUB[Publishers]
        SUB[Subscribers]
        PUBSUB[Pub/Sub Clients]
    end

    PUB --> BROKER
    BROKER --> SUB
    PUBSUB --> BROKER

    BROKER --> TOPICS
    TOPICS --> SUBS
    SUBS --> RETAINED

Topic Hierarchy¶

• Topics: Hierarchical structure with forward slashes (e.g., home/sensors/temperature)
• Wildcards: + (single level), # (multi-level)
• Retained Messages: Last message on a topic is retained for new subscribers
• Last Will and Testament: Message sent when client disconnects unexpectedly

Message Format¶

{
  "topic": "home/sensors/temperature",
  "payload": {
    "deviceId": "sensor-001",
    "temperature": 23.5,
    "humidity": 65.2,
    "timestamp": "2025-01-11T16:56:59Z",
    "location": "living-room"
  },
  "qos": 1,
  "retain": true,
  "messageId": 12345
}

Architecture Overview¶

Single Broker Architecture¶

graph TB
    subgraph "MQTT Broker"
        CORE[MQTT Core]

        subgraph "Components"
            AUTH[Authentication]
            TOPICS[Topic Management]
            SUBS[Subscription Manager]
            STORE[Message Store]
        end

        subgraph "Protocols"
            TCP[TCP/IP]
            WS[WebSocket]
            SSL[SSL/TLS]
        end
    end

    subgraph "IoT Devices"
        SENSOR[Sensors]
        ACTUATOR[Actuators]
        GATEWAY[IoT Gateway]
    end

    subgraph "Applications"
        MOBILE[Mobile Apps]
        WEB[Web Apps]
        BACKEND[Backend Services]
    end

    SENSOR --> TCP
    ACTUATOR --> TCP
    GATEWAY --> TCP

    MOBILE --> WS
    WEB --> WS
    BACKEND --> SSL

    TCP --> CORE
    WS --> CORE
    SSL --> CORE

    CORE --> AUTH
    CORE --> TOPICS
    CORE --> SUBS
    CORE --> STORE

Clustered MQTT Architecture¶

graph TB
    subgraph "MQTT Cluster"
        subgraph "Load Balancer"
            LB[Load Balancer]
        end

        subgraph "MQTT Brokers"
            B1[Broker 1]
            B2[Broker 2]
            B3[Broker 3]
        end

        subgraph "Shared Storage"
            DB[Database]
            CACHE[Redis Cache]
        end

        subgraph "Monitoring"
            MON[Monitoring]
            LOGS[Log Aggregation]
        end
    end

    subgraph "Clients"
        IOT[IoT Devices]
        APPS[Applications]
    end

    IOT --> LB
    APPS --> LB

    LB --> B1
    LB --> B2
    LB --> B3

    B1 --> DB
    B2 --> DB
    B3 --> DB

    B1 --> CACHE
    B2 --> CACHE
    B3 --> CACHE

    B1 --> MON
    B2 --> MON
    B3 --> MON

    MON --> LOGS

Target Operating Model (TOM)¶

Without High Availability¶

Single Broker Setup¶

Resource Requirements¶

Configuration Example¶

# Mosquitto broker configuration
port 1883
protocol mqtt

# Persistence
persistence true
persistence_location /var/lib/mosquitto/
persistence_file mosquitto.db

# Logging
log_dest file /var/log/mosquitto/mosquitto.log
log_type all

# Security
allow_anonymous true
password_file /etc/mosquitto/passwd
acl_file /etc/mosquitto/acl

# Limits
max_connections 1000
max_queued_messages 100

With High Availability¶

Cluster Setup¶

Resource Requirements (Per Node)¶

Deployment Architecture¶

graph TB
    subgraph "Production Environment"
        subgraph "Availability Zone 1"
            B1[Broker 1]
            LB1[Load Balancer 1]
        end

        subgraph "Availability Zone 2"
            B2[Broker 2]
            LB2[Load Balancer 2]
        end

        subgraph "Availability Zone 3"
            B3[Broker 3]
            LB3[Load Balancer 3]
        end

        subgraph "Data Layer"
            DB[PostgreSQL Cluster]
            REDIS[Redis Cluster]
        end

        subgraph "Monitoring"
            PROM[Prometheus]
            GRAF[Grafana]
            ALERT[AlertManager]
        end
    end

    subgraph "Edge Devices"
        IOT[IoT Devices]
        SENSORS[Sensors]
        GATEWAYS[Gateways]
    end

    IOT --> LB1
    SENSORS --> LB2
    GATEWAYS --> LB3

    LB1 --> B1
    LB2 --> B2
    LB3 --> B3

    B1 --> DB
    B2 --> DB
    B3 --> DB

    B1 --> REDIS
    B2 --> REDIS
    B3 --> REDIS

    B1 --> PROM
    B2 --> PROM
    B3 --> PROM

    PROM --> GRAF
    PROM --> ALERT

HA Configuration¶

# HiveMQ cluster configuration
cluster {
  enabled = true
  node-id = "node-1"

  transport {
    type = "tcp"
    bind-address = "0.0.0.0"
    bind-port = 7800
  }

  discovery {
    type = "static"
    static {
      node-addresses = [
        "node-1:7800",
        "node-2:7800",
        "node-3:7800"
      ]
    }
  }
}

# Persistence
persistence {
  type = "file"
  file-persistence {
    enabled = true
    storage-directory = "/opt/hivemq/data"
  }
}

# Security
security {
  allow-empty-client-id = false
  payload-format-validation = true

  authentication {
    type = "file"
    file = "/opt/hivemq/conf/credentials.xml"
  }
}

Pros and Cons¶

Pros¶

Lightweight & Efficient¶

• Low Overhead: Minimal protocol overhead
• Battery Friendly: Designed for low-power devices
• Bandwidth Efficient: Optimized for slow networks
• Small Footprint: Minimal resource requirements

IoT Optimized¶

• QoS Levels: Three quality of service levels
• Retained Messages: Last message retained for new subscribers
• Last Will: Automatic notification of client disconnection
• Keep-Alive: Heartbeat mechanism for connection monitoring

Flexible & Scalable¶

• Topic Wildcards: Flexible subscription patterns
• Hierarchical Topics: Organized topic structure
• Session Persistence: Persistent client sessions
• Bridge Support: Broker-to-broker communication

Industry Standard¶

• OASIS Standard: Internationally standardized protocol
• Wide Adoption: Extensive IoT ecosystem support
• Mature Ecosystem: Many broker implementations available
• Interoperability: Cross-platform compatibility

Cons¶

Security Limitations¶

• Basic Security: Limited built-in security features
• No Encryption: Plain text by default
• Simple Auth: Basic authentication mechanisms
• Limited Authorization: Topic-level access control

Reliability Constraints¶

• Broker Dependency: Single point of failure
• No Guaranteed Delivery: Even with QoS 2
• Limited Durability: Message retention depends on broker
• Connection Management: Requires careful connection handling

Scalability Challenges¶

• Vertical Scaling: Limited horizontal scaling options
• Memory Usage: High memory usage for many connections
• Topic Explosion: Performance degradation with many topics
• Broker Bottleneck: Centralized broker architecture

Feature Limitations¶

• Simple Protocol: Limited advanced messaging features
• No Transactions: No transactional message support
• Limited Routing: Basic routing capabilities
• No Load Balancing: No built-in consumer load balancing

Best Practices¶

Production Deployment¶

1. Security Implementation
2. Enable TLS/SSL encryption
3. Implement proper authentication
4. Use access control lists (ACLs)

5. Regular security audits

6. High Availability

7. Deploy multiple brokers
8. Use load balancing
9. Implement clustering

10. Plan for failover scenarios

11. Monitoring & Maintenance

12. Monitor broker performance
13. Track connection metrics
14. Set up alerting

15. Regular backup procedures

16. Topic Design

17. Design hierarchical topic structure
18. Use meaningful topic names
19. Avoid topic explosion
20. Plan for scalability

Development Guidelines¶

1. Connection Management
2. Implement reconnection logic
3. Handle connection failures gracefully
4. Use appropriate keep-alive settings

5. Manage client sessions properly

6. QoS Selection

7. Choose appropriate QoS levels
8. Understand delivery guarantees
9. Balance reliability vs performance

10. Handle duplicate messages

11. Message Design

12. Keep messages small and efficient
13. Use appropriate data formats
14. Implement message versioning
15. Handle message ordering

When to Choose MQTT¶

Ideal Use Cases¶

• IoT Applications: Device-to-cloud communication
• Telemetry Systems: Sensor data collection
• Mobile Applications: Push notifications
• Remote Monitoring: Industrial monitoring systems
• Home Automation: Smart home devices

Consider Alternatives When¶

• High Throughput: Millions of messages per second
• Complex Routing: Advanced routing requirements
• Enterprise Integration: Complex business logic
• Stream Processing: Real-time analytics
• Guaranteed Delivery: Strong consistency requirements