Libove Blog

Personal Blog about anything - mostly programming, cooking and random thoughts

Weekly 2022-34

Published:

How To Convert a List of Objects to CSV in JavaScript

Published:

This is a pretty common task in frontend; you already have some data (retrieved from an API) and want to offer the same data as a CSV.

The problem: most of the top answers you find on Google and StackOverflow are wrong. They will break in the presence of , or " in the data. Additionally all I found are vulnerable to CSV injections.

Usage:

  • yourData is the data you want to convert to CSV.
  • In data you have to change two things
    • Change the array to list all headers you want to be included in your CSV
    • Change the value lookups in the body to match the headers.
  • csvStr will be the finished CSV content. You can use this to construct a download for the user.

Snippet:

const yourData = [
	{value_a: 12, value_b: "some data"},
	{value_a: null, value_b: "no data"},
]

const escape = (str) => {
	// handle empty cell
	if(str == null || str == '') return '""'
	// ensure string
	str = `${str}`
	// prevent CSV Injection https://owasp.org/www-community/attacks/CSV_Injection
	const forbidden = new Set(["=", "+", "-", "@", "\t", "\n", "\r"])
	if (forbidden.has(str[0])) {
		str = `'${str}`
	}
	// escape double quotes
	str = str.replace(/"/g, '""')
	return `"${str}"`
};

const data = [
	// header
	[
		"Header A",
		"Header B",
	].map(escape),
	// body
	...yourData.map(
		row => [
			row["value_a"],
			row["value_b"],
		].map(escape)
	)
]
const csvStr = data.map(x => x.join(",")).join("\n")

console.log(csvStr);

What is CSV

The naive assumption about CSV (comma separated values) is just appending data with commas as separators in between. This works in some special cases, but for general data this scheme will quickly break. When you have data which includes commas it will break your CSV.

In  [1, 2, 'hello, world']
CSV 1,2,hello, world
Out [1, 2, 'hello', ' world']

To avoid this quotes are used. Either all or only the values where it is needed are put into quotes. The reader will ignore all commas if they are inside quotes. This way we can represent data that includes commas.

In  [1, 2, 'hello, world']
CSV "1","2","3","hello, world"
Out [1, 2, 'hello, world']

But this only shifts the problem. What happens if our quote character is used in our data?

In  [1, 2, 'hello", John," what?']
CSV "1","2","3","hello", John," what?"
Out [1, 2, 'hello', ' John', ' what?']

To avoid this an escape character is used. The escape character is placed in front of every quote char, which is part of the data. This indicates to the reader that the following quote is part of the data. The most common format of CSV uses the quotechar itself as the escapechar. Every " in the data is simply doubled to "".

In  [1, 2, 'hello", John", what?']
CSV "1","2","3","hello"", John"", what?"
Out [1, 2, 'hello", John", what?']

The combination of "separator", "quotechar" and "escapechar" allows us to encode arbitrary data as CSV, without breaking the format of our data. In principle anything can be chosen for these three characters. One common variation is to use a semicolon (;) as the separator. If a tab (\t) is used, the files are called "TSV" (Tab separated values).

Weekly 2022-33

Published: 
  app:
    logging:
      options:
        max-size: "10m"
        max-file: "3"

Weekly 2022-31

Published:

Weekly 2022-30

Published: 
user_comment = get_object_or_404(
    self.user.comments,
    pk=self.kwargs["comment_pk"],
)

Weekly 2022-29

Published:

Schneller Veganer Gnocchi Brokkoli Auflauf

Published:
Servings: 4 , Prep Time:

Ingredients

  • 800g Gnocchi
  • 1 Brokkoli
  • 1 Dose Tomatenstück
  • 1 Becher veganes Creme Fraiche
  • 1 Zwiebel
  • 1-2 Knoblauchzehen
  • Veganer Reibekäs
  • Olivenö
  • Oregano
  • Thymian
  • Salz und Pfeffer
  • Optional: Paniermehl und Leinensamen

Instructions

Schneller Veganer Gnocchi Brokkoli Auflauf.

Ein schneller veganer Auflauf mit Gnocchi und Brokkoli. Kann eingefroren oder backfertig im Kühlschrank gelagert werden, für Tage wenn man keine Lust hat zu kochen.

  • Wasser in Topf zum Kochen bringen und Gnocchi nach Packung kochen
  • Brokkoli in mundgerechte Stücke schneiden
  • Zwiebel und Knoblauch hacken
  • Zwiebel in Öl anbraten bis sie anfangen Farbe zu bekommen
  • Brokkoli und Knoblauch hinzufügen und kurz mit anbraten
  • Mit Tomaten ablöschen und Dose mit etwas Wasser ausspülen und mit hinzugeben
  • Für ein paar Minuten aufkochen, dann Creme Fraiche hinzufügen
  • Nach Geschmack würzen
  • Fertige Gnocchis in die Pfanne geben und alles gut vermischen
  • In Auflaufform geben und mit Käse bedecken
  • Optional etwas Paniermehl und Leinensamen über den Käse sträuen, für einer knusprigere Kruste
  • für ca. 20 Minuten bei 200°C backen

How to install an AppImage

Published:

In general AppImages can just started by double clicking or executing them from the terminal (./MyApp.AppImage). If you want to have them accessible from the start menu, you have to do a little bit more work.

First, move your AppImage to a suitable location. One option is to create a new directory for all your apps (e.g. an "Applications" directory in your home folder). I prefer to store them in .local/bin, as this folder is added to the $PATH variable. This allows you to use the app anywhere by calling it in a terminal. For GUI apps you most likely want to start them from the start menu of your desktop environment. For most apps, this can be achieved by following these steps.

  1. Open the App
  2. In a terminal call mount and look for the application name. You can use grep to filter the output. (mount | grep MyApp)
  3. Open the location in your file manager. In the directory you should find:
    • an icon. This can be a PNG in the main folder. In my case (Obsidian) the icons where stored in usr/share/icons/hicolor
    • a "desktop configuration file". It's content should look similar to this:
    [Desktop Entry]
Name=Obsidian
Exec=AppRun --no-sandbox %U
Terminal=false
Type=Application
Icon=obsidian
StartupWMClass=obsidian
X-AppImage-Version=0.13.19
Comment=Obsidian
MimeType=x-scheme-handler/obsidian;
Categories=Office;
  4. Copy the icon into .local/share/icons. If you find a hicolor folder in the app directory, copy the entire folder over. (use Ctrl+H to show hidden files)
  5. Copy the desktop configuration file into the .local/share/applications/ directory in your home folder.
  6. Edit the copied desktop file.
    • Replace the app name in the Exec= (and TryExec=) statement with the full path to the AppImage. In the example file above: Exec=~/.local/bin/MyApp