This post explores some changes in Big Sur that affect UI scripting, specifically UI scripts that interact with the menu bar. I learned much of this while updating my Keyboard Maestro macro for Sidecar to work with Big Sur, so most of the information is presented in that context.
A brief introduction to UI scripting
UI (user interface) scripting is a really neat macOS feature that allows you to automate almost anything using AppleScript. Even if the app doesn’t expose a scripting interface, UI scripting works because it simulates button presses, keystrokes and other actions that you would perform if you were doing the task manually.
UI scripts use an application called System Events. You manipulate user interface elements by telling System Events to tell another process (application) to do something. For example, this code will click the Export Playlist… item in the Music app’s File menu:
Often, the hardest part of writing a UI script is figuring out how to tell the process what to click. If you want the script to click a menu item, you usually can’t tell it to click the menu item directly; as in the example above, you have to give it the path from the menu bar down to the item you want it to click:
click menu item "Export Playlist..." of menu "Library" of menu item "Library" of menu "File" of menu bar 1. (This example is actually written from the bottom up, but you can see that it includes the path from the menu bar —
menu bar 1 — to the menu item
In addition to application menus, System Events can click system menus such as the volume menu and the Bluetooth menu. My Keyboard Maestro macro for Sidecar for Catalina and Big Sur uses a UI script to activate Sidecar via the Display menu. The original version of the macro broke under Big Sur, which is what led me down this rabbit hole.
Scripting the menu bar in Catalina
In Catalina, the system menus (Bluetooth, volume and so forth) in the menu bar are controlled by the process SystemUIServer. (SystemUIServer also controls the system menus in High Sierra and Mojave. I don’t have an OS older than High Sierra, so I don’t know if it was the same in older versions.)
You can see in this screenshot of Script Debugger running under Catalina that the element hierarchy for the Displays Menu is System Events > SystemUIServer > menu bar 1 > menu bar item 17. Using the description property shown in the screenshot, we can make that a little more intuitive, and not have to worry about finding the menu bar item number, by calling it
menu bar item 1 of menu bar 1 whose description contains "Displays". There’s only one menu bar, so
menu bar 1 always refers to the menu bar. There’s only one system menu called “Displays Menu,” so
menu bar item 1 whose description contains "Displays" refers to the Displays Menu.
Because the Displays Menu hasn’t been clicked, its menu items aren’t visible, so Script Debugger can’t show you is the items in the Displays Menu. For that you need an app like UI Browser.
Using UI Browser, you can find the scriptable menu elements as you click the menu. In this case, I want to find “Geoff’s iPad” in the Displays Menu.
Once we know how to refer to the menu items, we can write the AppleScript code:
(You can use Script Editor, which is installed by default on every Mac, instead of Script Debugger, but it’s not as full-featured. Likewise, Accessibility Inspector is a less capable but free alternative to UI Browser. It’s included with Xcode, which is a free download from the Mac App Store.)
Changes in Big Sur
Big Sur introduced breaking changes to the system menus. The first change you can see in the screenshot below is that most of the system menus aren’t controlled by SystemUIServer. In Big Sur, they’re controlled by the ControlCenter process.
Second, there’s no Displays Menu. Like many system menu items, it’s part of the Control Center menu and not displayed in the menu bar by default. And it’s now called “Display” instead of “Displays Menu.”
You can move the Display menu to the menu bar by going to System Preferences > Dock & Menu Bar and setting the Display menu to always display in the menu bar.
And now the menu bar contains a Display menu that shows my iPad as a Sidecar device.
You would think that at this point, we should be able to script the Big Sur menu bar in roughly the same way we did in Catalina. But there’s yet another big change: The system menus aren’t menu elements, and menu items aren’t menu item elements. In Big Sur, system menus are application windows, and menu items are buttons or checkboxes that are nested in groups and scroll areas.
Much like you have to specify the full menu hierarchy to make the script click a menu item, you must give the script the full path to the desired button. A note here on the different ways certain elements are identified: “Control Center” is identified as a system dialog and a window. My iPad is identified as a toggle button and a checkbox. The names in parentheses after the element (window and checkbox in this case) are the names you use in the script.
Putting it all together
Now that we know how the menus work in Big Sur, how do we put it all together to control the menu with a script? Step one is to determine if the Display menu is present in the menu bar. (I couldn’t get it to work unless the Display menu was always shown in the menu bar.) To accomplish this part, I took the inelegant approach of initializing the variable
displayMenu to an empty string (
""). Then the script checks all the menu bar items for one whose name contains ‘Display’. If it finds the Display menu,
displayMenu becomes a reference to the menu.
The script clicks the Display menu, which allows the script to “view” the hierarchy of elements and find the checkbox with the same name as the
deviceName variable. Then it clicks the checkbox, which activates Sidecar. (Admittedly, this section needs some error handling because the script will crash if the device isn’t in range of the Mac. But as long as the device is in range, it does the job.)
There’s one more accommodation we need to make for Big Sur. In Catalina, activating Sidecar closes the Displays Menu. In Big Sur, the Display menu stays open after activating Sidecar. We can close the menu by simply clicking it again, except that the menu name changes to “Display, Sidecar on” when Sidecar is active.
The script doesn’t know about the menu called “Display, Sidecar on,” so we use this code to correctly identify the menu and click to close it:
Here’s the full script:
The changes in Big Sur make it harder to properly identify the menu items for scripting, and Apple’s UI Scripting documentation — which is outdated and cursory — doesn’t help. Hopefully this post proves helpful in understanding the changes and adapting old scripts for Big Sur.