Setup Rabbitmq With Users for Docker Compose

This guide is written for RabbitMQ 3.9!

Configuring the RabbitMQ Server

For the configuration of the RabbitMQ server we will use a rabbitmq.conf and a definitions.json. I stored these files in a rabbitmq folder to keep my project folder structure clean. This setup is derived from sudos answer on StackOverflow

The rabbitmq.conf deactivates the default guest user and tells RabbitMQ to load the definition file.

loopback_users.guest = false
management.load_definitions = /etc/rabbitmq/definitions.json

In the definition file we can define our users, vhosts and permissions.

{
 "rabbit_version": "3.9",
 "users": [
  {
   "name": "local_jobs",
   "password_hash": ">>>HASH<<<",
   "hashing_algorithm": "rabbit_password_hashing_sha256",
   "tags": ""
  },
  {
   "name": "adminuser",
   "password_hash": ">>>HASH<<<",
   "hashing_algorithm": "rabbit_password_hashing_sha256",
   "tags": "administrator"
  }
 ],
 "vhosts": [
  {
   "name": "\/"
  },
 ],
 "permissions": [
  {
   "user": "local_jobs",
   "vhost": "\/",
   "configure": ".*",
   "write": ".*",
   "read": ".*"
  }
 ],
 "parameters": [],
 "policies": [],
 "queues": [],
 "exchanges": [],
 "bindings": []
}

To add the configurations to the RabbitMQ server they are added via the volumes options in the docker-compose.yml

  rabbitmq:
    hostname: rabbitmq
    image: rabbitmq:3.9-management
    command: rabbitmq-server
    ports:
      - "5672:5672"
      - "15672:15672"
    volumes:
      - ./rabbitmq/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
      - ./rabbitmq/definitions.json:/etc/rabbitmq/definitions.json:ro

Generating Password Hashs

In order to generate the password hashs I used the python script by Todd Lyons on StackOverflow

#!/usr/bin/env python3

# rabbitMQ password hashing algo as laid out in:
# http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2011-May/012765.html

from __future__ import print_function
import base64
import os
import hashlib
import struct
import sys

# This is the password we wish to encode
password = sys.argv[1]

# 1.Generate a random 32 bit salt:
# This will generate 32 bits of random data:
salt = os.urandom(4)

# 2.Concatenate that with the UTF-8 representation of the plaintext password
tmp0 = salt + password.encode('utf-8')

# 3. Take the SHA256 hash and get the bytes back
tmp1 = hashlib.sha256(tmp0).digest()

# 4. Concatenate the salt again:
salted_hash = salt + tmp1

# 5. convert to base64 encoding:
pass_hash = base64.b64encode(salted_hash)

print(pass_hash.decode("utf-8"))

Wahlprogramme

Ich habe ein kleines Tool für die Bundestagswahl 2021 gebaut um nachzugucken wie oft Begriffe in den Wahlprogrammen der Parteien vorkommen.

Die Suche ermöglicht es mehrere Begriffe in Relation zueinander darzustellen.

  • Begriffe werden durch Kommas getrennt (z.B. Klima,Wirtschaft)
  • Begriffe können durch Pluszeichen addiert werden (z.B. Klima+Umwelt)
  • Um nach einem exakte Begriff zu suchen kann dieser in Anführungszeichen gesetzt werden (z.B. "Klima")

The Cost of Proof of Work

Introduction

In the last months I’ve spent a lot of my time researching and thinking about the ecological impact of blockchain. Bitcoin and Ethereum, the two most popular blockchain technologies, have a vast power consumption. The power consumption is not inherent to blockchain technologies, but a result of the way these blockchains are secured.

The network of computers running Bitcoin and Ethereum establish trust by making changes to the blockchain expensive. This is achieved by requiring to solve a puzzle whenever you want to append something to the chain. Miners (people who solve these puzzles) have to spend money on hardware and electricity to solve the puzzles. The difficulty of the puzzle is adjusted automatically with the number (and power) of computers trying to solve the puzzle. Without adjusting the difficulty, changes would become inexpensive and the blockchain would become vulnerable. This method of securing a blockchain is called “Proof of Work” (PoW).

There are other methods to establish trust in a blockchain network, like Proof of Stake (PoS), which require far less energy. I want to focus on Proof of Work as this is the system which currently secures the most value in the crypto currency space.

It’s hard to calculate the real power consumption of a blockchain. Estimates are mostly concerned with the current hash rate (how many tries to solve the puzzle are executed each second) and the efficiency of the used hardware. It is almost impossible to know how efficient the hardware used is which leads to large ranges in the estimation. As of writing Bitcoins energy usage probably lies somewhere between 43 TWh and 477 TWh per year. In the following I will not concern myself with the efficiency of the hardware. Instead I try to derive the energy consumption of any blockchain (using PoW) from the value of the coins and how much miners are paid in block rewards.

Why miners mine

The reason why miners do their job of building and operating mining pools is simple: they want to earn money. For each block that is found by a miner she earns some coins.

There are generally two ways miners are rewarded for their work:

First the miner can create some new coins for each found block. The block rewards can change over time, for example bitcoin started with 50 BTC per block and the reward is lowered gradually until, eventually, no new coins are added to the blockchain. This mechanism is used to create the initial pool of coins and to incentivise mining while very little transaction fees are available.

When the block reward is diminished over time transaction fees play an important role to incentivise mining. When you create a transaction to be added to the blockchain a transaction fee can be added. This fee is given to the miner who created the block containing the transaction. Once no more coins are generated out of thin air the transaction fees have to pay the bills of the miners.

The block reward gives us an upper limit of the expenses of all miners. If 10 BTC are earned on average per block, all miners combined can at most spent 10 BTC to create a new block. If more resources were spent on creating a block over a longer period of time miners would go bankrupt.

Why efficiency does not matter

Miners have two main costs: hardware and electricity, everything else is overhead. In order to be profitable they constantly try to minimize the price per computed hash. The number of blocks generated by a miner is directly proportional to the percent of the global hash rate the miner controls. A miner holding 1% of the global hash rate will be able to create 1% of the blocks. As hardware gets more efficient, in terms of hashes per energy usage, miners will upgrade their hardware to minimize their cost and to secure more blocks. When other miners upgrade their hardware too the advantage of a single miner is diminished as the improved hash rate is lost when everybody increases their hash rate. This leads to a cycle of constant upgrade to stay ahead of the curve. The upgrade cycle only leads to a higher global hash rate but does not lower the power consumption of the networks as a whole as miners only try to secure a bigger piece of the block reward cake.

If you look at mining from above, ignoring the single miner, you see a collective that constantly buys more efficient hardware but somehow manages not to reduce its power consumption. Instead the power consumption of the collective is dictated by the rewards themself.

Higher rewards for mining, either by coins becoming more valuable or more fees being paid, lead to investments into mining equipment. Existing miners buy more machines and new people start mining. This leads to a higher energy consumption of the mining collective.

Any mining reward not used for new hardware (or kept as profit) will be used to buy energy to run the machines.

How much energy will be used

This gives us four factors that control the energy usage of all miners.

  • Block Reward: How much money can be earned by creating a block (Coin/Block)
  • Exchange Rate: What is the value of a coin ($/Coin)
  • Proportional Energy Costs: Which portion of the mining rewards is used for electricity (%)
  • Electricity Price: How much does an average miner pay for a kilowatt-hour ($/kWh)

The used energy of the network per block can be computed as:

([Block Reward] * [Exchange Rate] * [Proportional Energy Costs]) / [Energy Price]
 = [Energy Usage per Block]

Abstracting away blocks and exchange rates this could be simplified to:

a * [Daily Rewards] / [Energy Price] = [Daily Energy Usage]

where a is the proportion of the rewards used for energy

Blockchain Energy Usage Calculator

(The calculation assumes that a new block is generated every 10 minutes)

Revenue per Block: {{revenue}} $

Energy per Block: {{block}} GWh

Energy per Year: {{year}} TWh

Conclusion

The energy consumption of a PoW blockchain is proportional to the value a miner can earn per block. If the value of the reward doubles, either by higher fees or higher coin values, the energy consumption will double eventually. Similarly the power consumption will rise if the price of energy drops.

A widely used blockchain, which potentially could replace fiat currencies, would have a high transaction amount leading to a large pool of fees to be collected by miners. This would eventually lead to a corresponding energy consumption.