b1c1l1 blog

Linux Home Router Download Bufferbloat Analysis

2020-04-19T12:54:24-07:00

Introduction

In my previous article, Linux Home Router Traffic Shaping With FQ_CoDel, I mentioned that download shaping with rate limiting can avoid download bufferbloat. In this article, I wanted to take a closer look at the factors that contribute to download bufferbloat and how rate limiting works around the problem.

Background

Transmission Control Protocol (TCP) uses congestion control algorithms to adapt to network conditions. TCP senders increase the window size to use all available bandwidth until congestion is detected. However, detecting remote congestion is difficult in practice and the techniques for doing so are continuing to evolve.

Traditionally, TCP congestion control algorithms (e.g. Reno, CUBIC) assumed that congestion causes packet loss. Bufferbloat breaks this assumption by queueing packets, rather than dropping them, during periods of congestion. Despite being delayed, these packets are eventually delivered and ACKed, preventing senders from detecting and adapting to the congestion.

Some TCP congestion control algorithms (e.g. Vegas, BBR) can detect congestion by measuring delay. However, some concerns have been raised regarding how bandwidth is shared between flows using loss-based algorithms and flows using delay-based algorithms. Google created and uses TCP BBR in production.

Explicit Congestion Notification (ECN) extends IP and TCP by enabling congested network nodes to signal congestion explicitly by marking packets rather than dropping them. However, ECN adoption has been slow due to misbehaving devices dropping packets with ECN bits set. Most servers now support ECN for clients that request it, but most clients still do not enable ECN by default. FQ_CoDel enables ECN marking by default.

Download Bufferbloat

Download bufferbloat is difficult to analyze because it is highly variable and depends on many factors outside of our control. These factors include, but are not limited to: TCP congestion control algorithm, ECN support, round-trip time (RTT), traffic shaping, network congestion.

In particular, TCP congestion control algorithms run on the sender, which in the case of downloads is the remote server. If the server is configured to use a delay-based congestion control algorithm such as BBR, which is designed to reduce latency despite the presence of bufferbloat, then it will be especially difficult to measure and reproduce bufferbloat-related problems.

Rate Limiting

One workaround for download bufferbloat is rate limiting. Depending on how it is implemented this may also be called ingress shaping. An example implementation is available in my previous article.

Using rate limiting to avoid download bufferbloat is counterintuitive because it involves dropping valid packets that will need to be retransmitted. It also seems inefficient to sacrifice a significant amount of download bandwidth and leave the link underutilized.

It's worth reiterating that rate limiting is a workaround. When senders send us too much data, our options for asking them to slow down are limited. Although some senders might respond to delay and/or ECN, nearly all senders will respond to packets dropped due to rate limiting.

Here is a simplified visualization of the download bufferbloat problem and how rate limiting works around the problem. This example is based on traditional TCP congestion control algorithms such as Reno using additive increase multiplicative decrease. As the window size increases, download bandwidth is exceeded and packets are buffered upstream, causing latency. Using rate limiting to drop downloaded packets early ensures that window size increases above steady state remain below the actual link speed.

Caveats

Ideally, upstream devices would implement active queue management on their transmit queues to us and no download shaping would be necessary on our end.

Download shaping is at best a workaround for well-behaved senders that respond to packet loss by slowing down. Download shaping will have limited value in other scenarios such as floods, and may even be counterproductive if packet loss results in amplification.

Additional Reading

Linux Home Router Traffic Shaping With FQ_CoDel

2020-03-26T19:12:37-07:00

Background

Bufferbloat refers to excessively large network buffers causing high latency.

Many operating systems are configured by default to enable good performance on low-latency high-bandwidth links, such as data centers and corporate LANs. For example, with Gigabit Ethernet (1 Gbps) 1,500-byte packets can be sent at a rate of approximately 81,274 packets/sec. At this rate, a buffer of 1,000 packets represents a delay of ~12 ms.

1,000,000,000 bits/sec × 1 byte/8 bits × 1 packet/1,538 bytes ≈ 81,274 packets/sec

1,000 packets × 1 sec/81,274 packets × 1,000 ms/sec ≈ 12 ms

However, at low bandwidths, such as the 1 Mbps upload speed offered by my home ISP, a 1,000-packet buffer would represent a massive 12,000 ms delay. Worse still, by the time the tail of the queue is reached, it is unlikely that those packets still need to be delivered, since the users that sent them have already retried or given up.

Traditional QoS (Quality of Service) systems use packet metadata such as port numbers and TCP flags to identify high priority traffic. For example, an administrator can prioritize SIP traffic on port 5060 for VoIP calls or reserve bandwidth for TCP ACKs. However, these systems are complex to administer and are difficult to mantain as applications and requirements change. Additionally, these systems are becoming less relevant due to the increasing amounts of opaque traffic transported over HTTPS and VPNs.

Home internet connections exacerbate the bufferbloat problem due to having asymmetric links with low upload bandwidths in addition to poorly configured modems. Home users also have increasing amounts of data being uploaded, such as photos, video conferencing, and security cameras.

CoDel and FQ_CoDel

CoDel (Controlled Delay) is an Active Queue Management algorithm that drops packets based on delay rather than queue length. FQ_CoDel (Fair Queueing Controlled Delay) combines CoDel with fair queueing to share bandwidth between flows. CoDel was introduced in a 2012 publication, Controlling Queue Delay. The initial Linux implementations of sch_codel.c and sch_fq_codel.c were committed soon after.

Although CoDel is designed to be a "no-knobs" algorithm, tuning is required for low-bandwidth links. For example, at 1 Mbps transmitting a 1,500-byte packet takes ~12 ms, so the CoDel target should be increased from the default 5 ms. Additionally, the Bufferbloat Wiki recommends disabling ECN for uplinks less than 4 Mbps.

Implementation

The simplest implementation is to create a single HTB bucket with FQ_CoDel on the "WAN" interface connected to the modem. The rate limit must be lower than the available upload bandwidth (~95%) to ensure that packets are not buffered by the modem, however you will have to experiment to find the right value for your connection.

# tc qdisc add dev eno1.2 root handle 1: htb default 1
# tc class add dev eno1.2 parent 1: classid 1:1 htb rate 950kbit quantum 1514
# tc qdisc add dev eno1.2 parent 1:1 fq_codel target 15ms noecn

If download shaping is required, you can use a similar configuration on the "LAN" interface connected to your home devices. Download bufferbloat is a complex subject and will be discussed further in a future article. Usually you will have to use a significantly lower rate limit for downloads (~85%). If you have significant download traffic originating from the router, you may want to consider ingress shaping with an ifb device instead.

# tc qdisc add dev eno1.1 root handle 1: htb default 1
# tc class add dev eno1.1 parent 1: classid 1:1 htb rate 13mbit quantum 1514
# tc qdisc add dev eno1.1 parent 1:1 fq_codel

Alternatives

Some users may want to perform additional shaping by creating multiple HTB buckets with different rate limits and classifying packets using nftables, iptables, or tc filters. There are many examples of these kinds of configurations, however I do not think it is necessary for most use cases.

Alternatively, you might want to consider using CAKE, which adds interesting features such as pre-configured priority queues, fair queueing across LAN hosts when using NAT, and ACK filtering.

Testing

One way to test your configuration is to run ping while saturating your bandwidth (e.g. uploading a large file). If you are suffering from bufferbloat, ping will show significant delays and drops. I also recommend the DSLReports speed test and Fast.com speed test because they include tools for measuring latency.

Here is an example of running ping while saturating a 1 Mbps upload connection:

64 bytes from 1.1.1.1: icmp_seq=256 ttl=55 time=2944 ms
[dropped packet icmp_seq=257]
64 bytes from 1.1.1.1: icmp_seq=258 ttl=55 time=3109 ms
64 bytes from 1.1.1.1: icmp_seq=259 ttl=55 time=3211 ms
64 bytes from 1.1.1.1: icmp_seq=260 ttl=55 time=3235 ms

Based on the observed 3,000 ms delay and assuming a FIFO queue we can estimate that the modem transmit queue length is ~250 packets.

After implementing traffic shaping with FQ_CoDel, ping shows an improvement:

64 bytes from 1.1.1.1: icmp_seq=306 ttl=55 time=27.3 ms
64 bytes from 1.1.1.1: icmp_seq=307 ttl=55 time=25.9 ms
64 bytes from 1.1.1.1: icmp_seq=308 ttl=55 time=28.8 ms
64 bytes from 1.1.1.1: icmp_seq=309 ttl=55 time=33.1 ms

Much better!

Additional Reading

Encrypted Recursive DNS with DNS over TLS, Unbound, and Cloudflare

2018-04-23T14:17:35-07:00

The recent announcement of Cloudflare's new privacy-focused recursive DNS service 1.1.1.1 prompted me to revisit the options for encrypted recursive DNS and finally enable DNS over TLS on my workstations.

As a brief reminder, Transport Layer Security (TLS) and its predecessor Secure Sockets Layer (SSL) improve privacy online by using symmetric encryption to prevent eavesdropping and using public-key cryptography to authenticate recipients.

Due to increasing awareness and new revelations regarding sophisticated spying operations by governments and corporations, there has been a significant increase in the adoption of HTTPS (HTTP over TLS) over the past several years. According to Google, over 68% of Chrome traffic on Android and Windows is now protected.

However, recursive DNS, which is used before an HTTPS connection can be established, remains largely vulnerable to the same attacks that are used against unencrypted HTTP.

Fortunately, there are several options for protecting recursive DNS, including DNS over TLS, DNS over HTTPS, and DNSCrypt. Each option has various tradeoffs and may not be available depending on your specific operating system and internet provider. For example, you will need to ensure that the relevant ports are not blocked.

Unbound is a DNS resolver with native support for DNS over TLS. In version 1.7.0 you can use the appropriately named forward-tls-upstream option, while the same option was named forward-ssl-upstream in version 1.6.0. Older versions supported the global ssl-upstream option.

To set up a forwarding resolver with Unbound, simply create a forward-zone entry with name "." and the appropriate upstream addresses. Cloudflare supports DNS over TLS on 1.1.1.1 and 1.0.0.1 on port 853. The final result in your unbound.conf should look something like this:

server:
    tls-cert-bundle: /etc/ssl/certs/ca-certificates.crt

forward-zone:
    name: "."
    forward-tls-upstream: yes
    forward-addr: 2606:4700:4700::1111@853#cloudflare-dns.com
    forward-addr: 2606:4700:4700::1001@853#cloudflare-dns.com
    forward-addr: 1.1.1.1@853#cloudflare-dns.com
    forward-addr: 1.0.0.1@853#cloudflare-dns.com

It is necessary to specify tls-cert-bundle to enable TLS certificate verification and the forward-addr suffix '#cloudflare-dns.com' to specify the expected subject name on the certificate.

Additional reading:

Loading the GeoIP Module with nginx 1.10.0 on FreeBSD

2016-05-03T04:56:46-07:00

After upgrading to nginx 1.10.0 on my FreeBSD machines, I found that nginx would no longer start.

$ sudo /usr/local/etc/rc.d/nginx restart
Performing sanity check on nginx configuration:
nginx: [emerg] unknown directive "geoip_country" in /usr/local/etc/nginx/nginx.conf:42
nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed

Looking at the commit logs led me to revision 414084, which notes that the GeoIP module and several other modules are now built as dynamic modules.

So I looked for where the GeoIP module was installed.

$ pkg info -l nginx | fgrep geoip
        /usr/local/libexec/nginx/ngx_http_geoip_module.so

And updated the configuration to load the GeoIP module dynamically.

load_module "/usr/local/libexec/nginx/ngx_http_geoip_module.so";

Additional reading:

Introducing Dynamic Modules in NGINX 1.9.11

Disk Encryption with dm-crypt and LUKS on Linux

2015-09-20T19:27:55-07:00

dm-crypt is a Linux Device-mapper target that provides transparent encryption of block devices using the kernel crypto API. Plain dm-crypt supports only one passphrase and does not store any metadata on-disk.

Linux Unified Key Setup (LUKS) is a dm-crypt extension and the preferred way to set up disk encryption with dm-crypt. LUKS is a disk encryption standard that adds a header and a key-slot at the start of the device, called a LUKS container. LUKS supports multiple passphrases that can be individually changed and revoked. Passphrases are protected using the PBKDF2 key derivation function, which is also described in RFC 2898.

First, securely erase the target device.

For traditional hard disk drives, simply overwrite the device. Depending on your security requirements you may need to use random data instead of zeros.
```
cat /dev/zero > /dev/sdz
```
For solid-state drives (SSDs), clear the memory cells. This can usually be accomplished via ATA Secure Erase but varies by device. I do not recommend overwriting an SSD, which would increase device wear and cause reduced write performance due to write amplification.
```
sudo hdparm --user-master u --security-set-pass password /dev/sdz
sudo hdparm --user-master u --security-erase password /dev/sdz
```

Next, set up a LUKS device with your desired encryption parameters. The appropriate level of encryption is situation-dependent. I highly recommend choosing algorithms for which your system can use hardware acceleration (for example, the Intel AES-NI instruction set that was introduced with Westmere). Choosing cipher and hash algorithms is discussed at length elsewhere and is beyond the scope of this article.

Initialize a LUKS device and set the initial passphrase. For the block cipher I chose XTS-AES with a 256-bit key (XTS-AES 128), which is recommended in NIST SP 800-38E and used in Apple FileVault 2. For the PBKDF2 passphrase hash I chose SHA-256 with 5 seconds of processing time.
```
sudo cryptsetup --cipher aes-xts-plain64 --key-size 256 --hash sha256 --iter-time 5000 --use-random --verify-passphrase luksFormat /dev/sdz
```
Open and map the encrypted device. Use --allow-discards if using an SSD that supports TRIM.
```
sudo cryptsetup --allow-discards luksOpen /dev/sdz secure
```
Create and mount a filesystem on the encrypted device. Use -o discard if using an SSD that supports TRIM.
```
sudo mkfs.ext4 /dev/mapper/secure
sudo mount -o noatime,discard /dev/mapper/secure /secure
```

Finally, configure your system to map and mount the encrypted device during boot. These steps vary by distribution and whether you are encrypting the root filesystem, which requires updating your initramfs.

Here is an example configuring a non-root encrypted device on Gentoo Linux.

Add the LUKS device to /etc/conf.d/dmcrypt.

target="secure"
source="/dev/sdz"
options="--allow-discards"

Add the dmcrypt service to the boot runlevel.
```
sudo rc-update add dmcrypt boot
```

Add the encrypted filesystem to /etc/fstab.

/dev/mapper/secure      /secure         ext4            noatime,discard 0 2

Additional reading:

MacBookPro8,2 Kernel Patches for Linux 3.1.1

2011-11-19T13:53:39-08:00

I have run Gentoo Linux on multiple generations of the MacBook Pro since its introduction in 2006:

MacBookPro1,1
MacBookPro5,3
MacBookPro8,2

However, Apple uses bleeding edge and sometimes proprietary hardware in their products, which can be problematic for users of operating systems other than Mac OS because Apple does not provide drivers for those operating systems. This is especially true with the latest generation of hardware, which includes the second generation Intel Core processors, codenamed Sandy Bridge, with on-socket integrated Intel HD Graphics.

Consequently, I have compiled a set of patches for the Linux kernel that improves support for the 15-inch MacBookPro8,2:

blee@supra ~ $ sudo dmidecode -s system-product-name
MacBookPro8,2
blee@supra ~ $ lspci -nn
00:00.0 Host bridge [0600]: Intel Corporation 2nd Generation Core Processor Family DRAM Controller [8086:0104] (rev 09)
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port [8086:0101] (rev 09)
00:01.1 PCI bridge [0604]: Intel Corporation Sandy Bridge PCI Express Root Port [8086:0105] (rev 09)
00:02.0 VGA compatible controller [0300]: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller [8086:0126] (rev 09)
00:16.0 Communication controller [0780]: Intel Corporation Cougar Point HECI Controller #1 [8086:1c3a] (rev 04)
00:1a.0 USB Controller [0c03]: Intel Corporation Cougar Point USB Universal Host Controller #5 [8086:1c2c] (rev 05)
00:1a.7 USB Controller [0c03]: Intel Corporation Cougar Point USB Enhanced Host Controller #2 [8086:1c2d] (rev 05)
00:1b.0 Audio device [0403]: Intel Corporation Cougar Point High Definition Audio Controller [8086:1c20] (rev 05)
00:1c.0 PCI bridge [0604]: Intel Corporation Cougar Point PCI Express Root Port 1 [8086:1c10] (rev b5)
00:1c.1 PCI bridge [0604]: Intel Corporation Cougar Point PCI Express Root Port 2 [8086:1c12] (rev b5)
00:1c.2 PCI bridge [0604]: Intel Corporation Cougar Point PCI Express Root Port 3 [8086:1c14] (rev b5)
00:1d.0 USB Controller [0c03]: Intel Corporation Cougar Point USB Universal Host Controller #1 [8086:1c27] (rev 05)
00:1d.7 USB Controller [0c03]: Intel Corporation Cougar Point USB Enhanced Host Controller #1 [8086:1c26] (rev 05)
00:1f.0 ISA bridge [0601]: Intel Corporation HM65 Express Chipset Family LPC Controller [8086:1c49] (rev 05)
00:1f.2 SATA controller [0106]: Intel Corporation Cougar Point 6 port SATA AHCI Controller [8086:1c03] (rev 05)
00:1f.3 SMBus [0c05]: Intel Corporation Cougar Point SMBus Controller [8086:1c22] (rev 05)
01:00.0 VGA compatible controller [0300]: ATI Technologies Inc Whistler [AMD Radeon HD 6600M Series] [1002:6741]
01:00.1 Audio device [0403]: ATI Technologies Inc Device [1002:aa90]
02:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme BCM57765 Gigabit Ethernet PCIe [14e4:16b4] (rev 10)
02:00.1 SD Host controller [0805]: Broadcom Corporation NetXtreme BCM57765 Memory Card Reader [14e4:16bc] (rev 10)
03:00.0 Network controller [0280]: Broadcom Corporation BCM4331 802.11a/b/g/n [14e4:4331] (rev 02)
04:00.0 FireWire (IEEE 1394) [0c00]: Agere Systems FW643 PCI Express1394b Controller (PHY/Link) [11c1:5901] (rev 08)

These patches will also be useful for users of the 13-inch MacBookPro8,1 and 17-inch MacBookPro8,3 but I have not tested those models because I do not have access to them.

The following patches are for Linux 3.1.1:

apple_bl-gmux.patch
The display backlight on the MacBookPro8,2 is controlled by the Apple gMux. This patch adds Apple gMux support to the apple_bl driver to enable control of the display backlight.
The following kernel parameters must be specified to use the Apple gMux to control the display backlight:
```
acpi_backlight=vendor apple_bl.use_gmux=1
```
Related links:
- http://ubuntuforums.org/showpost.php?p=10694990&postcount=259
apple_gmux.patch
The MacBookPro8,2 includes two GPUs: the integrated Intel HD Graphics 3000 and the discrete ATI Radeon HD 6750M. In Mac OS, the discrete GPU is activated when an external monitor is connected or certain graphics-intensive applications are launched. This patch adds a driver for the Apple gMux to enable switching between GPUs using the vga_switcheroo mechanism. This patch is unnecessary when booting into BIOS emulation mode because the UEFI firmware prevents the integrated GPU from being accessed in BIOS emulation mode.
Related links:
- http://ubuntuforums.org/showpost.php?p=10848581&postcount=456
b43-ht-phy.patch
The Broadcom BCM4331 wireless chip in the MacBookPro8,2 is currently unsupported in the mainline kernel. This patch adds support for HT-PHY devices, including the BCM4331, to the b43 driver. I generated this patch by pulling the latest versions of the b43 driver and the related modules, bcma and ssb, from the wireless-testing.git tree. The b43 driver is currently the only driver with BCM4331 support, but the brcm80211 driver may add BCM4331 support in the near future.
Related links:
- http://lists.infradead.org/pipermail/b43-dev/2011-August/001978.html

efifb-hi-res.patch

My MacBookPro8,2 includes a high-resolution 1680x1050 display, which meant that the hard coded display information in the efifb driver was not correct for my display. This patch updates the efifb driver with the proper display information for the high-resolution display.

The display information was obtained using DTrace in Mac OS:

blee@supra ~ $ sudo ioreg -lw0 | grep product-name | cut -d'"' -f4
MacBookPro8,2
blee@supra ~ $ sudo dtrace -qn 'BEGIN {
> boot_args = (struct boot_args *) (`PE_state).bootArgs;
> printf("FrameBufferBase: 0x%08x\n", boot_args->Video.v_baseAddr);
> printf("PixelsPerScanLine: %d\n", boot_args->Video.v_rowBytes/4);
> printf("HorizontalResolution: %d\n", boot_args->Video.v_width);
> printf("VerticalResolution: %d", boot_args->Video.v_height);
> exit(0);
> }'
FrameBufferBase: 0x90010000
PixelsPerScanLine: 1728
HorizontalResolution: 1680
VerticalResolution: 1050

Using UTF-8 (Unicode) on FreeBSD

2011-05-09T00:14:37-07:00

Unicode is a set of character encodings that are compatible with the Universal Coded Character Set (UCS) defined by ISO/IEC 10646. Unicode was designed to replace all previous character encodings such as the American Standard Code for Information Interchange (US-ASCII) and ISO/IEC 8859.

UTF-8, which is also described in RFC 3629, is a variable-length Unicode character encoding that is backwards compatible with US-ASCII. That is, all US-ASCII characters have the same encoding under both US-ASCII and UTF-8. Due to the widespread use of US-ASCII in computing environments, this backwards compatibility makes UTF-8 convenient to deploy and therefore a popular choice for multilingual computing environments.

FreeBSD, like many UNIX-based operating systems, is unfortunately not configured to use UTF-8 by default. This sometimes causes confusion about whether Unicode is supported on FreeBSD. Fortunately, it is easy to enable UTF-8 on FreeBSD.

Determine the appropriate UTF-8 locale for your language and country. locale(1) can be used to print the names of all available locales.
```
locale -a | grep '\.UTF-8$'
```

Update the charset, lang, and setenv attributes in login.conf(5). It is recommended that LC_COLLATE be set to C because some programs still require ASCII ordering in order to function correctly.

To enable UTF-8 on a system-wide basis, update the default login class in /etc/login.conf.

blee@eclipse ~ $ diff -u /usr/src/etc/login.conf /etc/login.conf
--- /usr/src/etc/login.conf     2011-03-10 13:48:59.000000000 -0800
+++ /etc/login.conf     2011-05-08 16:44:01.000000000 -0700
@@ -26,7 +26,7 @@
        :passwd_format=md5:\
        :copyright=/etc/COPYRIGHT:\
        :welcome=/etc/motd:\
-       :setenv=MAIL=/var/mail/$,BLOCKSIZE=K,FTP_PASSIVE_MODE=YES:\
+       :setenv=MAIL=/var/mail/$,BLOCKSIZE=K,FTP_PASSIVE_MODE=YES,LC_COLLATE=C:\
        :path=/sbin /bin /usr/sbin /usr/bin /usr/games /usr/local/sbin /usr/local/bin ~/bin:\
        :nologin=/var/run/nologin:\
        :cputime=unlimited:\
@@ -44,7 +44,9 @@
        :pseudoterminals=unlimited:\
        :priority=0:\
        :ignoretime@:\
-       :umask=022:
+       :umask=022:\
+       :charset=UTF-8:\
+       :lang=en_US.UTF-8:


 #

To enable UTF-8 on a per-user basis, update ~/.login_conf. This is useful on servers that you do not administer and therefore cannot make system-wide changes.
```
blee@eclipse ~ $ cat ~/.login_conf
me:\
        :charset=UTF-8:\
        :lang=en_US.UTF-8:\
        :setenv=LC_COLLATE=C:
```

If /etc/login.conf was modified, run cap_mkdb(1) to rebuild the login class capabilities database.
```
sudo cap_mkdb /etc/login.conf
```
Exit all existing sessions that have the old locale settings.

Verify that the new settings took effect by running locale(1).

blee@eclipse ~ $ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_COLLATE=C
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=

If applicable, make application-specific configuration changes to enable UTF-8. Note that this has become increasingly unnecessary as applications have begun respecting locale settings.
Restart all applications that were started with the old locale settings.

Finally, here is the obligatory screenshot.

An excerpt from Markus Kuhn's Unicode/UTF-8 demo in Irssi running in GNU Screen over OpenSSH in Konsole.

For additional reading I recommend the UTF-8 and Unicode FAQ for Unix/Linux, the Localization - I18N/L10N Usage and Setup chapter of the FreeBSD Handbook, Using UTF-8 with Gentoo, and the Gentoo Linux Localization Guide.

Upgrading PostgreSQL on FreeBSD

2011-01-01T23:04:01-08:00

I decided to upgrade my PostgreSQL database instances because the default version of PostgreSQL on FreeBSD has been changed from 8.2 to 8.4. The internal data storage format changes in every major release of PostgreSQL, so upgrades require dumping and restoring the database.

First, dump the existing PostgreSQL database. It is essential that no changes are made to the database after it is dumped or data will be lost. There are many ways to achieve this, such as using a firewall or modifying pg_hba.conf. My preferred method is to start a temporary PostgreSQL instance on a non-standard port.

Stop PostgreSQL.

sudo /usr/local/etc/rc.d/postgresql stop

Start a temporary PostgreSQL instance on a non-standard port to dump the database.
```
sudo -u pgsql -H pg_ctl start -D /usr/local/pgsql/data -o "-p 55555"
```
Use pg_dumpall(1) to dump the database from the temporary instance.
```
pg_dumpall -U pgsql -p 55555 > pg_dumpall.`date +%Y%m%d%H%M`.sql
```

Stop the temporary PostgreSQL instance.

sudo -u pgsql -H pg_ctl stop -D /usr/local/pgsql/data

Rename the PostgreSQL data directory, which will not be used by the new version of PostgreSQL.
```
sudo mv -v /usr/local/pgsql /usr/local/pgsql.`date +%Y%m%d%H%M`
```

Next, upgrade the PostgreSQL ports. It is necessary to define DISABLE_CONFLICTS because the CONFLICTS check does not account for the change in port origin and will cause the in-place upgrade to fail.

Upgrade the PostgreSQL client first because it is a dependency of the PostgreSQL server.

sudo portupgrade -o databases/postgresql84-client -m DISABLE_CONFLICTS=yes "postgresql-client-*"

Upgrade the PostgreSQL server.

sudo portupgrade -o databases/postgresql84-server -m DISABLE_CONFLICTS=yes "postgresql-server-*"

Rebuild all ports dependending on the PostgreSQL client in order to ensure dynamic linking consistency after the changes in shared library versions.
```
sudo portupgrade -rf "postgresql-client-*"
```

Finally, set up the new PostgreSQL database. As with the dump, it is essential to prevent access to database until it is fully restored, which I achieve by starting a temporary PostgreSQL instance on a non-standard port.

Initialize the new PostgreSQL database using initdb.
```
sudo /usr/local/etc/rc.d/postgresql initdb
```
Reinstall configuration files that are stored in the PostgreSQL data directory. Note that PostgreSQL configuration options are frequently changed so any existing configuration files may require modifications in order to work with the new version. Configuration files that are stored in the data directory include:
- /usr/local/pgsql/data/pg_hba.conf
- /usr/local/pgsql/data/postgresql.conf
- /usr/local/pgsql/data/server.crt
- /usr/local/pgsql/data/server.key
Start a temporary PostgreSQL instance on a non-standard port to restore the database.
```
sudo -u pgsql -H pg_ctl start -D /usr/local/pgsql/data -o "-p 55555"
```
Use psql(1) to restore the database dump to the temporary instance.
```
psql -U pgsql -p 55555 -d postgres -f pg_dumpall.201101012028.sql
```

Stop the temporary PostgreSQL instance.

sudo -u pgsql -H pg_ctl stop -D /usr/local/pgsql/data

Start PostgreSQL.

sudo /usr/local/etc/rc.d/postgresql start