Working with APFS Volume Groups

Printer-Friendly Version
Product: 
ccc5

Note: This is pre-release documentation that is primarily applicable to macOS Catalina, which will be available in September 2019.

When Apple introduced the APFS filesystem several years ago, it came with a new concept: the APFS container. All APFS volumes reside within a container, and the container resides within the disk's partitioning scheme. All volumes within a container share the space that is available to the container; separate APFS containers do not share space with each other.

In macOS High Sierra, Apple added the concept of roles to volumes. At the time there were only three roles, and these went largely unnoticed by the average user: Preboot, Recovery, and VM (virtual memory). These roles allow the system to identify specific volumes for specific purposes, and then treat the volumes in specific ways (for example, any volume with the above roles would be hidden by default and also not mounted by default).

The following graphic demonstrates a few of these APFS concepts:

APFS filesystem concepts

The partitioning scheme encompasses the entire physical disk. Within the partitioning scheme you can create one or more APFS containers, and within each container, you can create one or more APFS volumes. Unlike partitioning in the past, all of the volumes within the container share the space that is allocated to the container. In the example above, the three gray helper partitions, the System and Data volumes, and the "Other Volume" all have access to that 700GB chunk of storage. "Other Volume B" is in a separate container, though, and does not share space with the volumes in container "A". Normally a disk would not be partitioned in this manner, but it would be warranted, for example, if you wanted to maintain a clone of your startup disk on that same disk (e.g. for testing purposes by developers).

New concept: APFS Volume Groups

In macOS Catalina, Apple introduced another new concept to the APFS filesystem: volume groups. This is more of a conceptual grouping of volumes within an APFS container, not a new sub-structure. Apple also greatly expanded the number of roles available for APFS volumes (now there are 16 unique roles). When you upgrade to Catalina, your current macOS system volume is renamed, e.g. to "Macintosh HD - Data", its role is set to Data, and then a new volume is added to your startup disk's APFS container with the System role and simultaneously grouped with the Data volume. The two volumes within that group share special bonds and receive special treatment from the Finder and from each volume's filesystem. From the user perspective, these two volumes are treated as a single, unified volume. If you take a look at Disk Utility, however, you'll see the two volumes as distinct, separate items.

The Read-only System volume

Perhaps the single, largest change in macOS Catalina is the manner in which the System volume is mounted on startup – it's read-only. By mounting the volume read-only, it becomes impossible for attackers to make changes to the content of the macOS System volume. That doesn't mean that your Mac is 100% free from all possible attack vectors, rather it's just another line of defense against them.

The Data volume

You can think of the Data volume as a read-write "shadow" of the System volume. The Data volume contains all of your user data (e.g. your home folder, third-party applications), but also contains a handful of system components that can't reside on a read-only volume. For example, Apple has placed Safari on the Data volume, perhaps so it can be updated more frequently. The current startup disk's Data volume is mounted at a special mountpoint on the system. You can find it if you navigate in the Finder to Macintosh HD > System > Volumes > {Data volume name}. What you'll find there is a replica of the System volume's root-level folders. Within these folders are all of the system components that are still writable. Normally you won't see these items in the Finder, though, because the Finder visually mashes the content of the two volumes together to make them appear as a single volume. Also, the Finder won't list your Data volume alongside all of your other volumes – the Data volume is mounted but hidden.

Building bonds with firmlinks

To pull off the illusion of a single, unified volume, Apple added support to APFS for firmlinks. Like the name implies, a firmlink lies conceptually between a soft link and a hard link. That probably doesn't make them any more clear though (even for people familiar with soft and hard links!). A firmlink is described by Apple as a "bi-directional wormhole" between two filesystems. Let's take a look at the "Users" folder as an example – the Users folder at the root level of the System volume is actually a firmlink that points to the Users folder at the root level of the Data volume. If you attempt to navigate to the /Users folder on the System volume, you're actually going to see the content of the /Users folder on the Data volume. Likewise, suppose you're looking at a folder on your Desktop (so you're looking at the contents of the Data volume) and then you navigate upwards several levels. When you get to the parent of the "Users" folder, you're no longer looking at the Data volume, rather that firmlink has transported you back to the root level of the System volume.

There are about a couple dozen firmlinks on macOS Catalina that link various folders on the System volume to writable counterparts on the Data volume. If you're curious about these, you can find a complete list of firmlinks at /usr/share/firmlinks on your startup disk.

Finder shenanigans with the Applications folder

Firmlinks are mostly transparent, but there is one really noticeable exception: the Applications folder. The Applications folder at the root level of the System volume is a firmlink to the Applications folder at the root level of the Data volume, however, if you navigate to your startup disk > System > Volumes > Data > Applications, you'll notice that the bulk of the Applications are not there. Yet when you look at the Applications folder on the System volume, they are all there! The Finder applies some magic here. The read-only System Applications folder actually resides at System > Applications on the System volume, and when you open the Applications folder in the Finder, you'll see the aggregation of that folder and the Data volume's root-level Applications folder. To the average user, this is exactly what you expect to see, and that's great. However, you may notice that this same aggregation is not applied to other system volumes that your Mac is not currently booted from (e.g. your backup disk). On those volumes, if you open the root-level Applications folder on the visible System volume, you'll only see the content of the firmlink to the root-level Applications folder on the Data volume (i.e. no Apple applications, just your third-party applications and Safari). Rest assured, though, that all of the applications are backed up! You'll find them at System > Applications on the backup volume.

CCC will automatically convert your HFS+ formatted destination to APFS to support backing up a Catalina+ volume

Because macOS Catalina leverages volume groups for the startup volume, creating a bootable backup requires an APFS formatted destination volume. HFS+ is no longer an option for booting macOS starting with macOS Catalina. For your convenience, CCC will automatically convert your HFS+ formatted backup volume to APFS as necessary. This conversion is the same conversion that took place on your startup disk when you upgraded to High Sierra or Mojave, with one notable exception: CCC tells you that it's going to convert the destination, and gives you the opportunity to decline the conversion. The conversion is non-destructive — any data that you have on the destination volume will remain in place, the only thing that changes is the format of the volume.

Why might I not want to allow the conversion of my destination volume?

Typically there is no reason to decline the conversion. The conversion is non-destructive, and it's required for making a backup of the system. If your backup volume is dedicated to your CCC backup task, then converting the destination to APFS is the right choice.

However, if your destination volume is not dedicated to your CCC backup task, you should consider how the other uses of your destination might be affected by the conversion. For example, Time Machine is not currently compatible with APFS as a destination, so converting a destination volume that contains a Time Machine backup would break the Time Machine backup. CCC specifically avoids converting Time Machine backup volumes.

If you're using the destination volume for non-CCC work, and you cannot or prefer to not convert the volume to APFS, you can create a dedicated partition on your destination disk for CCC to use. To create the partition:

  1. Open Disk Utility
  2. Select your destination disk in Disk Utility's sidebar
  3. Click the Partition button in the toolbar
  4. Click the "+" button to add a partition to the disk
  5. Set the name and size of the partition to your preference
  6. Choose APFS as the format
  7. Click the Apply button

Encrypted volumes and APFS volume groups

Encrypted HFS+ volumes cannot be automatically converted to APFS, and CCC cannot create an APFS volume group using an encrypted APFS volume. When you select a Catalina+ startup disk as a source and an encrypted volume as a destination, CCC will disallow the selection and suggest that you erase or decrypt the destination volume. Erasing the destination volume is the simplest approach, and you can find detailed instructions for doing that here: Preparing a hard drive for use with Carbon Copy Cloner

If you're making a backup to an encrypted volume that you've backed up to before (e.g. on Mojave and previous OSes), you may not want to erase all of the data on that volume. You can decrypt the destination volume with one of the following methods:

- Boot from the backup volume, open the Security Preference Pane, disable FileVault

- Decrypt the volume in the Terminal application. E.g. for an HFS+ formatted destination:
diskutil cs decryptVolume "/Volumes/CCC Backup"

Or for an APFS-formatted destination:
diskutil ap decryptVolume "/Volumes/CCC Backup" -user your_username

Tags: