Libove Blog

Personal Blog about anything - mostly programming, cooking and random thoughts

Visualizing Data on Maps using matplotlib and geopandas

For visualizing the heating and cooling degree days of Germany I had to learn how to plot maps.

First Look

My data source was a bunch of CSV files, which I combined into a single DataFrame, including latitude and longitude columns. I've used geopandas to plot this data for a first visualization. The DataFrame can be turned into a GeoDataFrame to create geometry information from the longitude and latitude columns. It's important to specify the right coordinate reference system (CRS) of your data.

With this conversion the data can be plotted using the plot function of geopandas

df = load_my_dataframe(...)
gdf = gpd.GeoDataFrame(
    df, geometry=gpd.points_from_xy(df["Longitude"], df["Latitude"]), crs="EPSG:4326"
)
ax = gdf.plot(
    column="ValueToVisualize",
)

Simple visualization of geography data using points

Adding Boundaries

The shape of Germany is already recognizable, but some more context would be beneficial. For this reason I've added the borders of the German states to the plot. The shape files of all European countries can be downloaded from eurostat in various formats. I've used the "Polygon (RG)" version and the .shp format. The files contain the borders on different levels (country, state and regional borders). With a few lines the borders can be added to the plot.

border_file = "NUTS_RG_01M_2021_3035.shp/NUTS_RG_01M_2021_3035.shp"
eu_gdf = gpd.read_file(border_file)
eu_gdf.crs = "EPSG:3035"
gdf_de = eu_gdf[(eu_gdf.CNTR_CODE == "DE") & (eu_gdf.LEVL_CODE == 1)]
gdf_de.to_crs("EPSG:4326").boundary.plot(ax=ax, color="black")

visualization of geography data using points and state borders

Filling the Map

Using points to visualize this data is not the best approach. Instead I want to fill the entire map where each pixel is colored according to the nearest data point. This can be achieved by computing a Voronoi Diagram, where each data point is turned into a face. I've tried to use the geoplot library to compute the voronoi diagram, but it got stuck on my data. Luckily scipy also has a voronoi implementation, but it's a bit more work to use.

# create a box around all data
min_lat = df["Latitude"].min() - 1
max_lat = df["Latitude"].max() + 1
min_lon = df["Longitude"].min() - 1
max_lon = df["Longitude"].max() + 1
boundarycoords = np.array(
    [
        [min_lat, min_lon],
        [min_lat, max_lon],
        [max_lat, min_lon],
        [max_lat, max_lon],
    ]
)
# convert our data point coordinates to a numpy array
coords = df[["Latitude", "Longitude"]].to_numpy()
all_coords = np.concatenate((coords, boundarycoords))
# compute voronoi
vor = scipy.spatial.Voronoi(points=all_coords)
# construct geometry from voronoi
polygons = [
    shapely.geometry.Polygon(vor.vertices[vor.regions[line]])
    for line in vor.point_region
    if -1 not in vor.regions[line]
]
voronois = gpd.GeoDataFrame(geometry=gpd.GeoSeries(polygons), crs="EPSG:4326")
# create dataframe
gdf = gpd.GeoDataFrame(
    df.reset_index(),
    geometry=voronois.geometry,
    crs="EPSG:4326",
)

# plot borders
gdf_de.to_crs("EPSG:4326").boundary.plot(ax=ax, color="black")
# clip geometry to inside of map
clip = eu_gdf[(eu_gdf.CNTR_CODE == "DE") & (eu_gdf.LEVL_CODE == 0)]
gdf = gdf.clip(clip.to_crs("EPSG:4326"))
# plot data
gdf.plot(
    ax=ax,
    column="Monatsgradtage",
    cmap="Blues",
    legend=True,
)

Visualization using Voronoi diagram

Making it beautiful

The plot is almost done, I only want to clean it up a bit. As the axis don't serve any purpose here, I removed them. Additionally I've added a title and a small text to the bottom to include the source in the graphic.

ax.set_title(f"Heizgradtage 2020", fontsize=20)
ax.axis("off")
ax.text(
    0.01,
    0.01,
    "Quelle: https://www.dwd.de/DE/leistungen/gtz_kostenfrei/gtz_kostenfrei.html\nVisualisierung: Niko Abeler CC-BY-SA 4.0",
    ha="left",
    va="top",
    transform=ax.transAxes,
    alpha=0.5,
    fontsize=8,
)

Final Visualization


Visualisierung: Heizgradtage und Kühlgradstunden Deutschland

Heizgradtage werden im Energiemanagement benutzt. Um den Heizbedarf über mehrere Jahre zu vergleichen muss der Effekt des Wetters herausgerechnet werden. Ansonsten vergleicht man jediglich das Wetter, da in kalten Wintern mehr geheizt werden muss. Dazu zählt man die Grad Celsius unter einem Schwellwert (meist 15°C in Deutschland).

Als Beispiel: Ein Tag mit konstanter Temperatur von 5°C ergibt 10 Heizgradtage ((15°C-5°C) * 1 Tag). Ein Monat mit 10°C ergibt 300 Heizgradtage ((15°C-5°C) * 30 Tage)

Die Visualisierung zeigt die Heizgradtage der Jahre 2010 - 2023 in Deutschland. Je kälter (mehr Heizgradtage) es in einer Region ist, desto dunkler wird diese dargestellt. Die Visualisierung basiert auf den Daten des DWD.

Parallel zu den Heizgradtagen gibt es auch die Kühlgradstunden. Hier werden die Stunden über einem Schwellwert (hier 18°C) gezählt. Eine Stunde mit 30°C ergibt somit 12 Kühlgradstunden. In dieser Visualisierung werden heiße Regionen in dunklerot dargestellt.


Valley of Dunes

Valley of Dunes

Serenity

Serenity

Endurance

Endurance

Open Sourced: Fedi-Games

I've open sourced fedi-games, as previously mentioned here. The code is still quiet rough as this is a learning project for me. But maybe someone finds this useful to implement their own ActivityPub server. Or maybe someone wants to build some new games :)


Fediverse Games

In order to understand the ActivityPub protocol (at some point I want to make my blog available via ActivityPub) I've created a minimal server hosting game services for the Fediverse.

The service is currently hosted on games.rerere.org and provides three games:

  • Rock Paper Scissors
  • Tic-Tac-Toe
  • Bunkers, an artillery game

The games can be played by mentioning the service and and your opponent in a note. I've mainly tested these with Mastodon but they should also work with other fediverse apps.

Screenshot from the Bunkers games

I want to release the source code soon, but it still needs some clean up, documentation and testing to be useful to anyone. The server is written in go and if you want to build something similar I've used these libraries to help with the ActivityPub implementation:


#


#


Inheritance Pattern in Go

Go isn't an object oriented language and doesn't have inheritance. Sometimes I still want to model something in a similar way as I would in an object oriented languages. My blog software has different types of "entries", e.g. articles, images, recipes ... This is modeled by an interface Entry which has to be implemented by each type of post.

// truncated for simplicity
type Entry interface {
	ID() string
	Content() EntryContent
	PublishedAt() *time.Time
	Title() string
	SetID(id string)
	SetPublishedAt(publishedAt *time.Time)
}

One problem with this is, that I have to implement these functions for each type of entry. Many of these implementation will be identical leading to a high degree of code duplication. To avoid this I've created an EntryBase, which implements sensible defaults.

type EntryBase struct {
	id          string
	publishedAt *time.Time
}

func (e *EntryBase) ID() string {
	return e.id
}

func (e *EntryBase) PublishedAt() *time.Time {
	return e.publishedAt
}

func (e *EntryBase) SetID(id string) {
	e.id = id
}

func (e *EntryBase) SetPublishedAt(publishedAt *time.Time) {
	e.publishedAt = publishedAt
}

With this "base class" the specific implementations only have to implement functions which differ between entry types.

type Article struct {
	EntryBase
	meta ArticleMetaData
}

type ArticleMetaData struct {
	Title   string
	Content string
}


func (e *Article) Title() string {
	return e.meta.Title
}

func (e *Article) Content() model.EntryContent {
    // render content from markdown	
}