Hacker News Reader

lfy_googleon ARM Mac: Why I'm Worried About Virtualization

Nice, that's good news.

lfy_googleon ARM Mac: Why I'm Worried About Virtualization

Android Emulator developer here. In addition to what the other comments said about android emulation with hypervisors existing already for x86, we're also looking into the Hypervisor.framework API for Apple silicon.

It won't be a trivial task (hoping for pre-existing code to port over maybe?) but we have the other pieces like using Hypervisor.framework for x86 already, and being able to cross compile the other code for arm64, so that would be the only major task left.

On the subj. of better GPU support, it depends on what it's actually like using the drivers, but from previous experience with the GPUs and drivers shipped with macOS, there shouldn't be any special kind of trouble at least. We may have to use Metal if Apple also gets rid of opengl support on those new machines, but there are also existing translators for gles and vk to metal. The graphics hw itself, is actually the least of our worries due to how consistent the hardware is likely to be---we'd have to deal with a much fewer set of hw/driver quirks versus other host OS platforms.

lfy_googleon Android Studio 4.0

Replace your current emulator install; delete $ANDROID_SDK_ROOT/emulator then unzip it there so theres $ANDROID_SDK_ROOT/emulator again.

$ANDROID_SDK_ROOT, if not set, should be set to the SDK location in Android Studio > Preferences > Android SDK (SDK path will show up there).

Set it in your ~/.bash_profile too so it's there next time for running the emulator from the command line.

The AVD config.ini can be found in ~/.android/avd/nameofAvd.avd/config.ini .

lfy_googleon Android Studio 4.0

And here's the promised emulator build (from aemubins gmail account that we use to distribute debug binaries like this):

    https://drive.google.com/open?id=1sXEQcoiNpe3Lj8yeaKLasTjqlGQwy4Oi

If you still don't get symbols from that it should be possible to build the emulator itself on macos:

install repo from depot_tools https://www.chromium.org/developers/how-tos/install-depot-to...

    pip install absl-py
    pip install urlfetch
    mkdir emu
    cd emu
    repo init -u https://android.googlesource.com/platform/manifest -b emu-master-dev --depth=1
    repo sync -qcj 4
    cd external/qemu
    python android/build/python/cmake.py

    # run
    objs/emulator -avd avdName -no-snapshot

Debugging:

    # when it spins:

    # method 1: go to activity monitor -> sample process, send over the result

    # method 2: lldb
    lldb
    process attach --pid <pid-of-qemu-system-<arch>>
    bt all #send it over

    # method 3: guest process taking CPU
    adb shell top

lfy_googleon Android Studio 4.0

Yes, the x86 system image can run ARM code. Note that the speed will be much faster than running a full ARM emulated system image because only the guest userspace bits need to be translated from ARM to x86. System calls are still marshalled to x86 calls, so memory accesses are many many times faster because (there's no need to drop into emulating the MMU).

(Also AFAIK illegal instructions are still validated and trapped as SIGILL or SIGSEGV)

lfy_googleon Android Studio 4.0

Interesting, sounds like a live loop in some audio thread somewhere. What's your audio setup and AVD config.ini? I might be able to reproduce on my Mac.

lfy_googleon Android Studio 4.0

Thanks, I've read about this before but just went over it again in more detail yesterday with a fresh look.

So this is basically the simulator situation, but with easier management of which libraries the guest userspace can dlopen and files to read()/write().

It's much like current Android-on-ChromeOS capabilities where containers are used to isolate where the "guest" userspace libraries are stored, so that it's not necessary to interop well with things like the host version of libc for example.

However, the problems come when considering the interface to the host kernel and hardware. Here are just two of the showstoppers:

1. Android expects to run on a particular range of kernel versions and configs each release. Fidelity is sacrificed to run with a wide range of Linux host kernel configurations. It's also easy for components on the host system such as SELinux to interfere with guest operation (and Android itself expects to use its own version of SELinux...so which one wins in the end?).

2. Further customization in the guest userspace needs to be made to account for needing to interop with a regular Linux system; e.g., input/network/display will be much more code that touches various parts of guest userspace and potentially hurts fidelity versus the VM abstraction where they are fake hardware and no customization of guest userspace is needed.

There are also isolation issues that involve more delicate dances, such as how to prevent runaway resource usage in the container from hogging the whole system (VMs merely waste the #vcpus + RAM dedicated to them; while that can be a lot compared to the host, it's explicitly controllable).

These problems sound less serious on the surface versus porting the Android framework directly to the host OS, but in the end it's basically the same level of essential complexity; containers just let you remove the incidental complexity of guest userspace libraries leaking into your /usr/lib and interop w/ your filesyste.

And once we try to run on non-Linux systems we're back at square one needing to port all userspace code to the host OS (Unless you're running Docker on macOS/Windows in which case you'd be creating a VM again, sacrificing all the benefits of containers versus VMs while keeping the complex customizations).

This is probably why Microsoft is pushing WSL2, ChromeOS skipped Android 10 support and is looking into ARCVM, and anbox is still running Android 7.1.1 (w/ plans to update but skipping releases in the meantime).

lfy_googleon Android Studio 4.0

Yep, that basically sums up what we're looking into---a system image that still keeps up with the latest API levels and features introduced in the latest Android OS, but has minimal impact on resources.

lfy_googleon Android Studio 4.0

Thanks for the interest! We've been exploring this in limited ways; for example, https://github.com/google/android-emulator-container-scripts contains a set of Python/Docker/JS scripts for setting up WebRTC streaming of emulators remotely. More to come soon.

lfy_googleon Android Studio 4.0

Yes, you can use the new AMD Hypervisor or use Hyper-V/WHPX, depending on your needs:

https://developer.android.com/studio/run/emulator-accelerati...

(Edit: On Linux, KVM for AMD should work already; this advice is for Windows hosts only)

lfy_googleon Android Studio 4.0

I'm on the emulator team; we've considered this route before, but the hard part about maintaining a simulator is that the contact surface with the host OS is much larger, making it so that we would essentially have to port the current year's version of Android, with all java/C userspace APIs, to all wanted host OSes. This would require much greater resources dedicated to this porting than we're able to handle, and/or we'd need to make the hard choice of only maintaining certain Android versions and skipping certain ones completely. It would also be very difficult to maintain fidelity.

With full virtualization, the contact surface is restricted to a few low level bits in the kernel along with a few HALs/drivers that need to talk to the host for meaningful/fast I/O, like input/network/graphics, and everything else can be kept stock with no modification. This allows us to ship largely the same binaries that would go on a dedicated Android device and have it be able to run on windows/macos/linux easily, and it's how we've been able to keep up with the pace of yearly Android releases.

Edit: Oh and note that we are totally aware of and bummed out by emulator's increasing resource usage as android version bumps up, to the point we're afraid that the next one will finally be the one that really needs all resources of a modern PC; as such, we're looking into ways to minimize the cpu/ram/disk footprint of the system images, keeping only the bits that are actually needed for testing apps (with Google Play Services, and being better at maintaining/promoting more stock AOSP images in the case where GMS isn't needed)

lfy_googleon Android Studio 4.0

Hi, what host OS are you running? I'd like to be able to send over a build with symbols so we can profile it (or on macos use the process sampler). Let's also try `adb shell top` next time this happens; we found that when networking state changes, sometimes, the virtual radio gets in a bad state and starts spinning.

lfy_googleon Android Studio 4.0

Hi, Android 9 and 11 Google APIs/Play system images should have userspace ARM translation available (with ARM64 translation available in Android 11).

lfy_googleon Tinyalloc: replacement for malloc/free in unmanaged, linear memory situations

https://android.googlesource.com/platform/external/qemu/+/em...

These 3 files should be a fairly self contained, minimal way to allocate into existing buffers. No attempt is made at thread safety though (we assume the user synchronizes that themselves)

lfy_googleon Poka-yoke

There's also "The Machine That Changed the World":

https://www.lean.org/Bookstore/ProductDetails.cfm?SelectedPr...

Generally, anything auto manufacturing related works too.

Also:

"The Principles of Product Development Flow: Second Generation Lean Product Development"

https://www.goodreads.com/book/show/6278270-the-principles-o...

We see a lot of words written on "lean" software development but I feel like it's better to learn from broader sources, and from industries that have had a lot more regulation and time to develop good practices.

lfy_googleon Android Emulator – AMD Processor and Hyper-V Support

Android Emulator, which is based on a fork of QEMU, has used Hypervisor.framework by default for macOS 10.10+ since version 26.1.0 (last year).