Back to the dev blog Release notes · iPhone & iPad

SharkTTY 1.7 and 1.8: a floating keyboard, a magnifier, and the Mac’s sound in your hand

· Safafish

Two tags an hour apart on July 2: 1.7, then 1.8. That is not a mistake in the log. 1.7 is the feature release — the one where the remote desktop stopped being a viewer and became somewhere you can actually work — and 1.8 is 1.7 plus one thing: a real audio codec under the sound that 1.7 had just started sending. On the Mac side they pair with SharkTTY Desk 0.7.0 and 0.8.0.

A keyboard that is actually a keyboard

Controlling a Mac from an iPad with the iOS keyboard is an exercise in frustration. There is no Escape. There is no Control. There are no function keys. Everything macOS expects you to hold, iOS does not have.

So 1.7 grew its own. It started as a floating QWERTY board on the iPad remote desktop and then, over a dozen commits, turned into something you would recognise as a keyboard: symbols, Tab and Return; ⌃, ⌥ and ⌘ as real modifier keys with a wide Shift; an ISO-style L-shaped Return spanning two rows, which then moved down to the home and bottom rows with a two-cell Shift and an fn key once we had looked at where thumbs actually land.

The two settings that made it usable rather than merely present: you can resize it, and you can set its opacity on a slider — all the way to fully opaque if you would rather not see the desktop through it. A keyboard covering the bottom third of an iPad is a different proposition when you can see what is underneath it. And rather than two competing buttons, there is now one keyboard button with a mode dropdown: the floating board, or the system keyboard with a shortcut toolbar.

Pointing at a pixel with a finger

A fingertip covers roughly the area of a macOS window's three traffic-light buttons. The virtual mouse exists because of that arithmetic, and 1.7 gave it a magnifier — a loupe that shows the area under your finger at the Mac's native resolution, so you can land on a checkbox instead of near one. Both the pointer and the loupe were extended to work across the aggregated screen layout rather than only within a single display.

One fix in that area is worth spelling out, because it is the sort that only turns up in use. The loupe tile is a separate stream of pixels from the main picture and arrives fractionally later. While the cursor is moving, that means the magnified view shows where the cursor was. Showing a stale magnifier is worse than showing none, so it now hides while the cursor is in motion and comes back when it settles.

Screens as one picture you can arrange

1.6 put every display on screen. 1.7 let you arrange them: a merged-screens dropdown, and a display area you can move and resize. That last part needed a handle, and here is why. The same drag gesture means two different things depending on intent — move this picture, or move the pointer on the Mac inside it. Guessing between them is a bad experience in both directions, so moving and resizing are gated behind an explicit adjust mode with a handle, and pinch-to-zoom only applies there too.

Right-click had the same problem, in miniature. It had been done differently in different places; 1.7 made it a two-finger tap everywhere, in the tiled view and the focused one. Single displays also got inset from the top bar and the bottom edge, so the picture is not hiding under the app's own chrome.

Quality that is negotiated, not assumed

Video quality is now a conversation. The Mac agent advertises which tiers it can encode, the viewer picks one and renders it, and the ceiling — xHD, the Mac's native pixels — can be requested from the device. It sits on the capability-bit machinery added in 1.6, which is what lets a newer app ask an older agent for something and get a clear no rather than a broken stream.

Sound, in both directions

1.7 added system audio from the Mac to the device, and a microphone uplink going the other way, so an iPhone can be the microphone for a Mac that is somewhere else. Between them, a call taken on the Mac is a call you can actually take.

Which is where 1.8 comes in. Audio at that point was raw PCM on a data channel: correct, and far more bandwidth than sound needs. 1.8 added Opus — the codec vocabulary in the protocol with its own capability bit, libopus 1.5.2 vendored into the build with a safe wrapper around it, and viewer-side decoding wired through the FFI. The capability bit is the important half: an agent and a viewer that do not both speak Opus fall back to what they do share instead of failing. The remaining pieces of that path — a buffered encoder that turns arbitrary PCM into 20 ms Opus frames, and playback on the viewer — finished in the following build.

Meanwhile, in the terminal

The remote desktop took most of the attention in 1.7, but the terminal got three things worth having:

  • File upload straight from the accessory bar, so a config file gets to a server without a detour through a file manager.
  • A hide-keyboard button, for reading output on a phone screen that is mostly keyboard.
  • A mouse-reporting toggle. With it on, a connected trackpad or mouse sends its clicks and drags to programs that understand them — vim, tmux, htop, less. With it off, the pointer selects text instead. Both behaviours are right; only you know which one you want right now.

The floating keyboard also picked up symbols, Tab and Return on the terminal side, and was relaid out to match a standard US keyboard rather than an approximation of one — groundwork for the six real layouts that arrived in the next build.

Read this post in: English · 中文 · 日本語 · Español · Français · 한국어 · Українська

More on the app itself in the SharkTTY guide, the short user-facing notes in What's New, and the fastest way to reach us is the feedback board. Back to the dev blog.