NAME

Matrix is a specification (not a single app or company) for how independent servers can talk to each other to support chat, voice/video call signaling, and IoT device communication - all without any single company or server being in charge of the whole network. Think of it as similar in spirit to email: anyone can run a mail server, and servers from different providers can still deliver mail to each other. Matrix does the same thing for real-time messaging.

I got into this matrix stuff when I realized that LinkedIn is a big scam. Matrix is decentralised, that is no central figure owns it. It is kinda TOR, but in surface web - that is you don’t risk yourself from scammers that are in TOR nodes potentially, plus the matrix.org offers its own server, which is one of the biggest matrix servers in use. Another plus - you can even create your own matrix server, just follow its protocol.

I have read the specification for the matrix version 1.19 which can be found here. I just refined the same in more comprehensible, in my own words.

So the core idea is a decentralized publish-subscribe system: data (mostly JSON objects) gets published into “rooms,” and any server participating in that room receives and stores a copy, keeping everything in sync even though no central authority owns the conversation.

• Practical over pure - plain JSON over HTTP/REST, not some exotic new protocol
• Simplicity - minimal moving parts and dependencies
• Full openness - anyone can run a server and join the network (open federation), and the spec itself is public with no patent restrictions
• User control - you pick your own server and client software, you control your own privacy, and you know exactly where your data physically lives. Go to matrix.org, and then try matrix - you would get a list of matrix clients like Element, Element X, Element for web, FluffyChat, and several others.
• No central point of failure or control
• Learning from older protocols - deliberately borrowing good ideas from XMPP, SIP, IRC, SMTP, IMAP, and NNTP while trying to dodge the problems that plagued each of them, and that’s why I think I should use matrix. Moreover, there are no/less API restrictions, so it is easy for me to create matrix bots to send messages, log details, and retrieve them in a private group with only me and my bot.

The spec supports: creating and managing chat rooms with no single owner; keeping room history synced (eventually consistent) across many servers; sending messages with optional end-to-end encryption; a permission system for inviting/kicking/banning members based on power levels; customizable room metadata (name, topic, aliases); user profile data (avatars, display names); account management (register/login/logout); and linking third-party identifiers like email or phone numbers so people can find each other by info they already know, backed by a network of trusted “identity servers” that verify these mappings.

The long-term goal is for Matrix to be a general-purpose sync layer for any structured data between people, devices, or services - not just chat.

• Client-Server API - how your chat app talks to your homeserver: this is what I use for my bot management. You first create a bot account in matrix.org and create a private group for you and your bot. The bot’s homeserver is matrix.org, my primary homeserver may or may not be matrix.org - I may use any other server, thanks again to the Server-Server API, that is matrix.org and anything.org can communicate with each other though they are entirely different servers. After creating and inviting the bot to the group (room), I just use the chat to send a message. Client-Server: the bot is the client, and matrix.org is the server for the bot. In the room, the message needs to reach my primary account - so matrix.org and anything.org will follow the Server-Server API to communicate with each other. Then Server-Client: anything.org will communicate with my primary client to make that message visible.
• Server-Server API - how homeservers talk to each other (this is what “federation” means): server-server is handled by the homeservers themselves, anything that follows the matrix protocol if configured can be added to the federation. So it is just a configuration to write.
• Application Service API - a privileged API for bots/bridges that need to act as many virtual users at once. This is exclusively designed for bots, but right now, I don’t need it - just me and my bot, so there is no need for virtual users.
• Identity Service API - for mapping emails/phone numbers to Matrix accounts
• Push Gateway API - for delivering mobile push notifications
• Plus specs for room versioning and the Olm/Megolm encryption algorithms used for E2EE

Every user has a homeserver - the server that stores their account and message history. When you send a message, here’s the actual path it takes:

1. Your client does an HTTP PUT to your own homeserver with the message content
2. Your homeserver adds this to its local copy of the room’s history, cryptographically signs it
3. Your homeserver forwards it via another HTTP PUT - this time over the Server-Server API - to every other homeserver that has users in that room: that is, if a room has 3 different users with 3 different homeservers, then it would forward it to all three homeservers.
4. Each receiving homeserver checks the signature, validates the content is allowed, and adds it to its own local copy
5. The other users receive the new message via a long-held GET request their client keeps open to their homeserver (this technique is called long-polling)

It is basically all PUT, POST, GET.

So a message you send doesn’t go directly to the recipient - it goes to your homeserver, gets replicated to their homeserver, and their homeserver hands it to them. No message ever needs a single “master” server to pass through.

One important tradeoff: Matrix deliberately prioritizes staying available and tolerating network splits over having perfectly consistent data everywhere instantly (this is a direct reference to the CAP theorem - you can’t fully have Consistency, Availability, and Partition-tolerance all at once, and Matrix picks the latter two).

Every account gets a user ID shaped like an email address: @username:servername. The part after the colon just tells you which homeserver issued the account - it doesn’t mean the conversation data lives only on that server. My bot can be like @mybot:lolcat.org, and primary be like @myname:somekind.com.

In Matrix, “device” doesn’t just mean a physical gadget. Every distinct login session - your phone app, your laptop’s browser tab, a second browser on that same laptop - counts as its own separate “device,” each with its own device_id. This matters primarily for end-to-end encryption: each device gets its own independent set of encryption keys, so you can revoke just one device (e.g., a stolen phone) without logging out everywhere else. Whether a device is long-lived or gets thrown away depends on the app - a website might create a fresh device every time you log in, while a mobile app might keep reusing the same one across sessions.

Device in this context means - everywhere you are logged in, that is each matrix session is a device. To stabilise that session, you would need a device id. That’s it.

Every single action in Matrix - sending a message, changing the room name, someone joining - becomes an event, which is just a JSON object with a type field describing what kind of event it is. Built-in event types defined by the spec always start with m. (like m.room.message); custom application-specific event types must use reverse-domain naming (like com.example.game.score) to avoid clashing with other apps’ custom types.

An important security note baked into the design: event content coming from the network should always be treated as untrusted input. A client or server must validate the shape of an event before trusting any of its fields, since there’s no absolute guarantee every expected field will actually be present or of the right type.

Rather than a simple linear timeline, each room’s history is stored as a directed acyclic graph (DAG) - every event points back to the event(s) that came immediately before it from that server’s perspective. Usually that’s just one prior event, but if two homeservers send messages at almost the same moment, you can get a temporary fork with two parents pointing at the same predecessor, which then merges back together as new events reference both branches.

To keep a sense of chronological order without needing a single global clock, every event carries a depth number - strictly larger than any of its parents’ depth values. This lets any server figure out relative ordering just by walking the graph, even under network delays. The very first event in any room has a depth of 1. This has something to do with graph theory stuffs, but I am not going in depth into this rabbit hole - for now, this is enough to get started with the API.

A room is identified by an opaque ID shaped like !randomstring:servername. That domain suffix is only there to prevent ID collisions globally - it does not mean the room is physically hosted on that one server. In federation, a room’s data is actually replicated across every homeserver that has a member in it (remember - it is replicated).

Room data splits into two categories:

• Message events - one-off, transient activity: a chat message, a call-setup signal, a file share. Say it is just like the messages sent in WhatsApp, like files sent, files received, etc in a group chat.
• State events - durable, current facts about the room: its name, topic, member list, which servers are participating. State works like a key-value table, where each entry’s key is a combination of an event type and a “state key,” and newer state events simply overwrite older ones for that same key. Say it is just like a user viewing the group info.

If two servers create conflicting state changes at nearly the same time (a race condition), Matrix runs a state resolution algorithm to deterministically decide which version wins - every server, regardless of the order it received events in, ends up agreeing on the same outcome.

Each event is cryptographically signed by the server that created it, and that signature covers the event’s parent links, type, depth, and content hash - so tampering is detectable. Servers propagate new events to each other directly (a full-mesh pattern, meaning every server in a room can talk to every other one), and can also request older history (“backfill”) from peers if they’re missing parts of it.

It is better to be read as #roomname rather than !oijup24UIOh, and that is why we have this.

Since room IDs are opaque strings, Matrix also supports aliases shaped like #name:servername - these are the human-readable names you’d actually see or type, like a bookmark. An alias simply points to an underlying room ID, and that mapping isn’t permanent - it can be repointed to a different room ID later. Because of that, clients are expected to resolve an alias to a room ID once and then keep using that ID going forward, rather than re-resolving on every request. Looking up an alias also tells you which servers are known to participate in that room, which is useful info for actually joining it.

A Matrix “identity” combines your Matrix user ID with any external identifiers (email, phone number, etc.) you’ve chosen to link to it. Third-party ID servers - a separate, trusted, federated network of “identity servers” - verify that you actually own that email or phone number and then store the verified mapping, so other users can find your Matrix account by searching for your email instead of needing your exact Matrix ID. This is entirely optional - you can use Matrix without ever touching an identity server, you just lose the ability to be found via 3rd-party contact info.

Users can publish public profile info (display name, avatar image URL, contact details) that others can see, and separately store private key-value data tied to their account (client settings, preferences) using a symmetrical but non-public API.

Namespacing: anything under the m. prefix is officially defined by the spec itself; anything else (custom event types, custom fields) must use reverse-domain naming to avoid two unrelated apps accidentally colliding on the same identifier.

Timestamps: everywhere in the spec, a timestamp means milliseconds since the Unix epoch (Jan 1, 1970 UTC), and leap seconds are deliberately ignored so every day is always treated as exactly 86,400,000 milliseconds - I mean this matches how most programming languages already represent time by default.

Matrix allows you to use simultaneous versions, that is you can use the protocol specifying what version of it you want to use, all with just a single version string in the API path.

The whole specification carries a single version number like vX.Y. A change to X signals something big and breaking (e.g., dropping JSON entirely, changing the cryptographic signing approach). A change to Y means new features added in a backwards-compatible way. There’s no guarantee that, say, v1.3 stays compatible with v1.1 - only that each step (v1.1 → v1.2) is meant to be safe.

Individual endpoints are versioned separately from the spec as a whole - so /v3/sync can be superseded by /v4/sync without touching or breaking /v3/profile, which keeps working exactly as it did.

Deprecation works in stages: a feature first gets marked deprecated (but servers/clients must still support it), then after roughly one spec version of being deprecated, it becomes eligible for full removal in a later version. If your software advertises support for a spec version that included a now-deprecated endpoint, you’re still required to implement it - you only get to drop it once you stop claiming support for that older version.

Legacy versioning: before this unified vX.Y numbering existed, each of the five APIs (Client-Server, Server-Server, Application Service, Identity Service, Push Gateway) was versioned completely independently using an rX.Y.Z scheme. What’s now called “Matrix 1.0” was never an official spec version number - it’s really shorthand for a specific combination of those older per-API version numbers, alongside room versions 1 through 5.

This is an example of bot script that I wrote using python for sending messages to a chat. Potentially it can be used for logging activities and sending files/photos, etc stuff and messages from a host machine to the room. It is based on a json file structure which has the access token to the bot account and room id to the room. This should kinda look like something like this :

{
    "name" : "lolcat-api",
    "bot" : {
        "username" : "some_random_bot",
        "homeserver" : "https://yourhomeserver.tld",
        "access_token" : "rct_euepJKSII0UUJvcvfIsL1L1Brw5Ehb_asds4",
        "device_id" : "0b90dB86UU",
        "user_id" : "@some_random_bot:matrix.org"
    },
    "rooms" : {
        "lolcat-api" : {
            "id" : "!OEpoYrzABCBiEWqe3l:matrix.org",
            "desc" : "A demo for learning matrix API and bot accounts for internet details redirection"
        }
    }
}

Note that the above json code is just the structure of how the json file should look like, nothing is real there.

Next we have this python code which will read and parse the json from the .data.txt file given above, but there is also a variable called CONFIG using which you can change the path of the data.json. For this purpose, we would be using the httpx module, that is more advanced and capable than the requests module in python. Here is the code :

#!/usr/bin/env python3

# import the necessary modules.
import sys
import os
import uuid
import json
import mimetypes
import httpx

# set the global config path.
CONFIG = ".data.json"  # adjust path if needed

# load the config path
def load_config(path=CONFIG):
    with open(path, "r") as f:
        return json.load(f)

# get the home server from the json file.
def get_homeserver(config):
    homeserver = config["bot"]["homeserver"]
    if not homeserver.startswith("http"):
        homeserver = f"https://{homeserver}"
    return homeserver

# define a function to send message to the server.
# for understanding what is happening beneath this function, you need to understand the matrix api.
def send_message(body, room_key, config, html=None):
    homeserver = get_homeserver(config)
    access_token = config["bot"]["access_token"]
    room_id = config["rooms"][room_key]["id"]

    txn_id = uuid.uuid4().hex
    url = f"{homeserver}/_matrix/client/v3/rooms/{room_id}/send/m.room.message/{txn_id}"

    payload = {"msgtype": "m.text", "body": body}
    if html:
        payload["format"] = "org.matrix.custom.html"
        payload["formatted_body"] = html

    resp = httpx.put(
        url,
        json=payload,
        headers={"Authorization": f"Bearer {access_token}"},
    )
    if not resp.is_success:
        sys.exit(f"Matrix API error {resp.status_code}: {resp.text}")

    return resp.json()


def upload_file(filepath, config):
    homeserver = get_homeserver(config)
    access_token = config["bot"]["access_token"]

    mimetype, _ = mimetypes.guess_type(filepath)
    mimetype = mimetype or "application/octet-stream"

    filename = os.path.basename(filepath)
    with open(filepath, "rb") as f:
        data = f.read()

    resp = httpx.post(
        f"{homeserver}/_matrix/media/v3/upload",
        params={"filename": filename},
        headers={
            "Authorization": f"Bearer {access_token}",
            "Content-Type": mimetype,
        },
        content=data,
    )
    if not resp.is_success:
        sys.exit(f"Upload failed {resp.status_code}: {resp.text}")

    return resp.json()["content_uri"], mimetype


def send_file(filepath, room_key, config, msgtype="m.file"):
    homeserver = get_homeserver(config)
    access_token = config["bot"]["access_token"]
    room_id = config["rooms"][room_key]["id"]

    mxc_uri, mimetype = upload_file(filepath, config)
    filename = os.path.basename(filepath)
    filesize = os.path.getsize(filepath)

    txn_id = uuid.uuid4().hex
    url = f"{homeserver}/_matrix/client/v3/rooms/{room_id}/send/m.room.message/{txn_id}"

    payload = {
        "msgtype": msgtype,  # m.file, m.image, m.audio, m.video
        "body": filename,
        "url": mxc_uri,
        "info": {"size": filesize, "mimetype": mimetype},
    }

    resp = httpx.put(
        url,
        json=payload,
        headers={"Authorization": f"Bearer {access_token}"},
    )
    if not resp.is_success:
        sys.exit(f"Matrix API error {resp.status_code}: {resp.text}")

    return resp.json()


def main():
    if len(sys.argv) < 4:
        sys.exit(
            "Usage:\n"
            '  python send-files-httpx.py msg <room-key> "message text"\n'
            "  python send-files-httpx.py file <room-key> <filepath> [--type m.image]"
        )

    action, room_key = sys.argv[1], sys.argv[2]
    config = load_config()

    if action == "msg":
        message = sys.argv[3]
        result = send_message(message, room_key, config)
        print(f"Sent message to {room_key}. Event ID: {result.get('event_id')}")

    elif action == "file":
        filepath = sys.argv[3]
        msgtype = "m.file"
        if "--type" in sys.argv:
            msgtype = sys.argv[sys.argv.index("--type") + 1]
        result = send_file(filepath, room_key, config, msgtype=msgtype)
        print(f"Sent file to {room_key}. Event ID: {result.get('event_id')}")

    else:
        sys.exit(f"Unknown action '{action}'. Use 'msg' or 'file'.")


if __name__ == "__main__":
    main()

# usage for this is :
# python3 send-files-httpx.py <msg-type:msg|file> <message|path/to/file> --type <if image, m.image>

How to actually use this script to automate the messaging, sending files and messages?

All we need is just two matrix accounts. (preferable) or even one matrix account (making your primary account act as bot). The difference between the two is that : the room is created by the bot account in first case and the primary account is invited to the room. The latter one, you just create a room and invite no one. You act as a bot and the script sends the message on behalf of you.

Step 1 : Create a primary matrix account in your preferred matrix server. Then procceed creating the bot account in case you don’t want to leak your primary account’s access token.

Step 2 : Figure out your access token. You can do it by just curling to a matrix API endpoint (this entire blog post will give you some kinda help). You can follow this oneliner -

# Run in Terminal/CLI
curl -X POST https://yourserver.tld/_matrix/client/v3/login \
     -H "Content-Type: application/json" \
     -d '{
           "type": "m.login.password",
           "identifier": {"type": "m.id.user", "user": "your_bot_username"},
            "password": "your_bot_passwd"
         }'

Step 3 : Replace the contents of the .data.json file from the results from your curl. You would need to create a room and get its room id. Replace the dummy id in the file with your original room id. Invite your primary account to the room.

Step 4 : Run the python script and you are good to go. See the last comment for usage.

I have not yet published to code to github. I need to document it and explain it, and do some advanced stuffs like defaulting, falling back and learn more of matrix APIs for doing automation and bots. See the official spec given below for more details.