Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GC: Support object level interop on Android #112109

Open
2 of 6 tasks
mangod9 opened this issue Feb 3, 2025 · 12 comments
Open
2 of 6 tasks

GC: Support object level interop on Android #112109

mangod9 opened this issue Feb 3, 2025 · 12 comments

Comments

@mangod9
Copy link
Member

mangod9 commented Feb 3, 2025

This is a tracking issue for support Android scenarios on CoreCLR GC.

  • Prototype using IReferenceTracker - repo
  • Prototype using bespoke solution - demo and bridge custom runtime
  • Implement the GC Bridge
    • Get new API approved
    • CoreCLR
    • native AOT
@mangod9 mangod9 added this to the 10.0.0 milestone Feb 3, 2025
Copy link
Contributor

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

@mangod9 mangod9 moved this to UserStories + Epics in Core-Runtime .net 10 Feb 3, 2025
@filipnavara
Copy link
Member

I did some experiment with running the MonoVM GC bridge code inside NativeAOT runtime - https://github.com/filipnavara/runtime/tree/javainterop. Notes: https://gist.github.com/filipnavara/17430066bdc3a9dce10ab65a46ae5dc4

Feel free to take any inspiration from it.

@jkotas
Copy link
Member

jkotas commented Feb 3, 2025

cc @AaronRobinsonMSFT

@cshung
Copy link
Member

cshung commented Feb 6, 2025

@filipnavara, I have read through the algorithm described in the paper.
I am wondering how exactly do we mirror the references once we have the compressed reference graph?

@AaronRobinsonMSFT
Copy link
Member

I am wondering how exactly do we mirror the references once we have the compressed reference graph?

This would follow the existing solution in Java interop. See add_reference and gc_cleanup_after_java_collection. The contract interface on the Java side is GCUserPeerable.

@filipnavara
Copy link
Member

filipnavara commented Feb 6, 2025

I was about to post what @AaronRobinsonMSFT did. However, there is a small nuance that is missing from the java-interop bridge and that is present in the Android bridge / MonoVM's sgen-tarjan-bridge. Sometimes it's inefficient to remove all C# objects from the compressed graph. For example, consider having a C# array AR = new JavaObject[1000] with each element pointing to a Java peer object, and three Java objects A, B, and C each pointing to the array AR and becoming collectible (as in, not having C# references; and having the C# projections becoming unreferenced). If you fully compress the graphs you end up with 3000 edges in the graph that need to be transferred to the Java VM. However, instead of doing that, you can represent the C# array as ArrayList on the Java side. By doing that you add 3 references to the ArrayList through the GCUserPeerable interface, and you create the ArrayList itself through JNI and simply abandon it at the end of the process. This essentially reduces the complexity from transferring 3000 edges to 1003 edges + 1 list allocation.

More generally, any node in the object graph that is pure C# object without Java peer can be represented as ArrayList on the Java side.

Just to summarize:

  • .NET projections of existing Java code (ie. wrapping existing unmodified Java class) cannot have references to other C# object unless the references are projected Java objects. For example, projection of Activity would fall in this category. It can reference implementations of ActivityLifecycleCallbacks that get attached through registerActivityLifecycleCallbacks, those implementations can be written in C# but they will themselves have Java projections.
  • .NET implementations of Java objects (including Java interface implementations, or objects inherited from existing Java classes) have the Java side of the projection generated by the dotnet/java-interop tooling and it implements the GCUserPeerable protocol on the Java VM side.
  • Any non-peer .NET node in the graph can be represented as ArrayList on the Java VM side.

@AaronRobinsonMSFT
Copy link
Member

However, there is a small nuance that is missing from the java-interop bridge and that is present in the Android bridge / MonoVM's sgen-tarjan-bridge.

I wasn't aware of this optimization. Can you share where the ArrayList creation is performed?

@filipnavara
Copy link
Member

Can you share where the ArrayList creation is performed?

https://github.com/dotnet/android/blob/627e239f8296ebd35101a80041fa6c95cbf8efd1/src/native/monodroid/osbridge.cc#L786-L818

@filipnavara
Copy link
Member

(Apparently the ArrayList is wrapped in a GCUserPeer class but that's an implementation detail.)

jonpryor added a commit to dotnet/java-interop that referenced this issue Mar 7, 2025
Context: dotnet/runtime#112109
Context: f5ce0ad
Context: 3824b97
Context: c6c487b

Commit f5ce0ad copied portions of the .NET for Android Mono GC Bridge
code into `src/java-interop`, but to *build* that code "on Desktop" --
for possible CI and unit test use -- it needed to build against Mono
*somehow*.

The *actual* "how" was unspecified:

  * The macOS build implicitly required that
    `/Library/Frameworks/Mono.framework` exist, and would pull in
    header files and libraries from there.

    This directory would only exist if you installed Mono system-wide
    on your Mac.

  * Linux and Windows were not support *at all*.

Eventually `src/java-interop` got Windows build support in 3824b97.
This allowed `Java.Interop-Tests.dll` to run under .NET Core, but
this did *not* contain any GC bridge code; it was *just* the JNI
wrapper functions such as `java_interop_jnienv_get_version()` and
JVM loading functions such as `java_interop_jvm_create()`.

Eventually (c6c487b), a native `java-interop` library was not needed
*at all*: C#9 function pointers and `NativeLibrary` removed the need
for it to be built at all.

The Mono GC bridge code within java-interop effectively bitrotted, as
nothing *built* it anymore.  .NET for Android has its own (original!)
copy, and CI is primarily concerned about .NET (Core) and not Mono.
(This means the GC bridge *isn't* tested, but everything else is.)

Three things have happened in the meantime:

 1. The Mono team was absorbed into the .NET team, with the new
    "MonoVM" as a supported backend for .NET, selectable by setting
    `$(UseMonoRuntime)`=true on desktop platforms.  MonoVM is used by
    .NET for Android and a supported backend with .NET for iOS, Mac…

 2. MonoVM is now provided as NuGet packages!

 3. There are now efforts to investigate adding something like the GC
    Bridge to CoreCLR and NativeAOT; see dotnet/runtime#112109.
    Some developers involved with this would like to debug how the
    current GC bridge works.  *On desktop Windows*.

Embrace the new world order: remove support for using "system" Mono
installations, the `_CreateMonoInfoProps` target, generation of
`MonoInfo.props`, etc., and instead use MonoVM packages to provide
the MonoVM runtime and header files needed for compilation & linking.
This allows for a consistent build environment across Linux, macOS,
and Windows, though Windows adds one wrinkle:

MonoVM doesn't contain a `coreclr.lib` file for linking.

Add `coreclr.def` and a `coreclr.lib` file to use for linking against
MonoVM on Windows.  `coreclr.lib` was created by:

	lib /def:coreclr.def /out:coreclr.lib /machine:X64

where `coreclr.def` contains the `mono_` symbols that were referenced
by the macOS `libjava-interop.dylib` build, as per
`nm bin/Release-net8.0/libjava-interop.dylib | grep 'U _mono'`.

Update `src/java-interop` so that it builds with the latest MonoVM
headers (`<mono/metadata/appdomain.h>` must now be included), and
allow it to build under Windows (use `Interlocked*` instead of
clang's `__sync_*_and_fetch()`.)

There is one ("tiny") problem with this approach: .NET 9 doesn't
contain MonoVM packages.  For example, the
[`Microsoft.NETCore.App.Runtime.Mono.osx-x64` NuGet package][0] was
last updated for .NET 9 Preview 7; the last *stable* version is for
.NET *8*.

Fortunately we only recently migrated to .NET 9 (f30e420).
Update `JniRuntime.cs` so that we can build the repo when overriding
with `-p:DotNetTargetFramework=net8.0`.

Update `Java.Interop.Sdk` so that the `.jar` it creates is
appropriately copied to `$(OutputPath)` of the referencing project.
This required using `%(Content.CopyToOutputDirectory)`=PreserveNewest
alongside  using `%(Content.TargetPath)` -- which sets the relative
name within `$(OutputPath)` -- and in turn required making
`_JavaCreateJcws` run after `CoreCompile` instead of `Build`, so that
it would properly participate in `%(CopyToOutputDirectory)` logic.

The code to detect that MonoVM is being used within
`Java.Runtime.Environment.dll` has changed; the `Mono.Runtime` type
no longer exists.  Check for `Mono.RuntimeStructs` instead.

Update `TestJVM.GetJvmLibraryPath()` so that it can find
`bin\BuildRelease\JdkInfo.props` when running an app from the
`samples` directory and not from `bin\TestRelease*\…`.

All of this allows for `samples/Hello-Java.Base` to be built against
and run with MonoVM, on Windows!

	# need Release config build because `dotnet publish` is implicitly Release
	dotnet build -c Release -t:Prepare -p:DotNetTargetFramework=net8.0 Java.Interop.sln
	dotnet build -c Release -p:DotNetTargetFramework=net8.0 Java.Interop.sln

	# -p:UseMonoRuntime enables use of MonoVM
	dotnet publish --self-contained -p:UseMonoRuntime=true -p:DotNetTargetFramework=net8.0 -p:UseAppHost=true -p:ErrorOnDuplicatePublishOutputFiles=false samples/Hello-Java.Base/Hello-Java.Base.csproj -r win-x64

Note: replace `-r win-x64` with the appropriate RID for your host.

With that setup, you can then run the sample:
	
	> samples\Hello-Java.Base\bin\Release\win-x64\publish\Hello-Java.Base.exe
	MonoVM support enabled
	# jonp: LoadJvmLibrary(C:\Users\jopryo\android-toolchain\jdk-17\bin\server\jvm.dll)=140710450692096
	# jonp: JNI_CreateJavaVM=140710454831248; JNI_GetCreatedJavaVMs=140710454831280
	# jonp: executing JNI_CreateJavaVM=7ff9b4ad2890
	# jonp: r=0 javavm=7ff9b51fd7f0 jnienv=24d781dddc0
	WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethodV
	binding? net.dot.jni.sample.MyJLO@a09ee92

[0]: https://www.nuget.org/packages/Microsoft.NETCore.App.Runtime.Mono.osx-x64/
@AaronRobinsonMSFT
Copy link
Member

@mangod9 This issue can be considered done in my opinion since both prototypes have been completed. The bespoke solution above using GCHandle is the approach I am recommending.

@mangod9
Copy link
Member Author

mangod9 commented Mar 19, 2025

Ok sounds good, should we continue using this issue to track progress on the actual work or create a new item for it?

@AaronRobinsonMSFT
Copy link
Member

We can use this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: UserStories + Epics
Development

No branches or pull requests

5 participants