类的结构

Java 语言中，类中的结构有字段（Field）、方法（Method）、构造器（Constructor）、参数（Parameter）、record component（RecordComponent）。

对字段、方法、构造器，Class 中有命名如 get(Declared)Xxxs 方法批量获得此类结构，例如 getDeclaredFields。

• 如果名称中包含 Declared，获得的结构不包含从上级类继承的，但是包括所有本类中定义的此类结构，包括非 public 结构。
• 否则返回的结构包括上级继承的结构（最优先本类定义结构，然后字段继承先接口（只有静态字段）再上级类，方法继承优先上级类再接口（接口静态方法无继承）详见 JVMS 5.4，构造器无继承），只包括 public 结构。（不包含从上级类继承的 protected 结构）

同时还有 get(Declared)Xxx 接收参数，精准获得符合条件的结构。字段额外接收字段名称，方法接收方法名称和参数类型数组，构造器接收参数类型数组。能返回的结构必定存在于 get(Declared)Xxxs 返回中，如果无符合则报错。

Parameter 可以通过 Executable::getParameters （方法和构造器的公共上级类中）获得，RecordComponent 可以通过 Class::getRecordComponents 获得。

这些方法返回数组，所以标准库每次返回时会复制一份数组。字段、方法、构造器还是 AccessibleObject 子类，这个类有 setAccessible，是可变对象，所以这些结构在标准库返回时也会被复制一次。为了避免额外开销，使用这些方法时最好获取一次数组或者结构，然后缓存返回的数组或结构重复使用，避免获取和复制开销。

这些结构适合用来获得结构上的 Java 语言类型（例如泛型）和注解。字段、方法、构造器还提供方法获得及改变字段值和呼叫方法和构造器；这些用途方便一次性使用，但如果需要多次使用这些功能，使用 MethodHandles.Lookup 获得对应的 MethodHandle 或者 VarHandle 更合适，因为这些呼叫和改变方法每次使用时都会进行权限检查，影响性能。详见 java.lang.invoke 包相关介绍。

Java 语言中的类型

Java 语言定义（Java Language Specification）第四章定义了 Java 语言中的类型，核心反射也有模型类，为了和已有的 Class 兼容，和语言中的类型有比较复杂的对应关系。

各种结构中获得类型的接口 getXxxType(s) 返回 Class 类型，同时有 getGenericXxxType(s) 返回 Type 类型，可能是列表中的某一个模型类。

标准库返回的类型模型对象不可变，但是数组也是可变对象，有额外复制开销，所以如果返回的对象需要重复使用，推荐缓存数组或模型对象重复使用。

注解

注解可以携带自定义数据，可以存在于定义（结构）上（declaration annotation）或类型的用途中（type(-use) annotation）。携带类型的模型类都实现 AnnotatedElement 接口。以上提到的结构和类型都可以携带注解。核心反射能发现的注解都要有 @Retention(RetentionPolicy.RUNTIME) 元注解。

注解也有 get(Declared)Annotation(s) 的区分，类似结构；注解继承只存在于类与接口上，影响很小。Java 8 允许注解重复，有特殊的 get(Declared)AnnotationsByType 可以处理重复注解。

类型用途使用注解有一套模型类，和语言中的类型模型类相似但有些地方有细微差别，例如 AnnotatedArrayType 模型包含 int[]，但在语言类型模型中不由 GenericArrayType，而由 Class 模型此类型。

核心反射提供的注解皆通过代理（Proxy）实现，可序列化。所以性能会有些劣势。因为返回的数组可变，有复制开销，推荐有需要缓存注解及其数组重复使用。

代理

为了方便 reflect 提供了一个代理工具，允许提供一个类加载器和一个接口列表，返回一个实现全部接口，类由指定类加载器加载的代理实例。代理的工厂方法还接收一个处理器，所有代理对象的呼叫（包括所有继承的抽象与非抽象接口方法和Object的equals、hashCode、toString方法）都会转到处理器。

这个工具过去常用于实现 AOP，现在推荐能编译时预处理可以考虑编译时预处理。它所有方法都要转到处理器，对 JIT 编译采样及其不友好，所以性能不堪入目；现在有新的类文件接口，条件允许可以生成自己的高性能动态类替代。

Note: This page is still under construction

CSR 意为兼容性和定义复核，是改动 JDK 接口必须经过的一个流程。这个流程相当于对接口和被依赖的行为的品控，避免对 Java 平台的一些改动带来过大恶果。

大致流程

• 提交主补丁，确定改动的 Javadoc 和 API 定义
• 在主 issue 下 More -> Create CSR，然后填写 CSR
• 别人审核批准补丁也会审核批准你的 CSR，算工程师批准
• CSR 获得工程师批准后从草稿状态转定稿
• 等 CSR 领头批准。
  • 批准时可能会提其他要求，比如改一些词汇或者写更新日志，不要忘
  • 有可能变临时，需要你回复疑问或者修改，然后重新转定稿，等下一轮批准
  • 批准后又对 Javadoc 或者 API 定义改动，则需要重新转草稿修改，然后再转定稿重新批准！
• CSR 通过后可以合并补丁

CSR 格式

CSR 都有主 issue。在主 issue 中选择 More -> Create CSR 即可创建空模板的新 CSR。

格式原文可参见 wiki

• Description 介绍：文字部分分４块，Summary 总结、Problem 问题、Solution 方案、Specificaiton 定义。
  • 总结：简单概括改动的主旨，比如添加新字段方法
  • 问题：列举下需要改动的理由，比如某些值常用但是直接写常量容易出错，最好用字段常量等，或者某些方法很常用，同时 JDK 能提供的实现比用户自己实现的更好
  • 方案：列举下具体的改动。有时候也可以列举下其他方案，然后为什么不采用其他方案等
  • 定义：定义变动。一般 Javadoc 都算定义，除了 @apiNote @implNote，然后接口类型字段加减也算。一般上传 git diff 但是移除非接口和定义改动。
• Assignee 负责人：只有负责人能够推进 CSR 的状态
• Component/Subcomponent：和主 issue 相同
• Status 状态：
  • Draft 草稿：刚创建的初始状态。
  • Proposed 提议：告诉 CSR 希望获得早期审核，一般会给反馈后进临时。
  • Provisional 临时：审核后的状态。记得回复或者进行改动，然后重新进终稿！
  • Finalized 定稿：有了工程师批准后其他状态才可以进定稿。只有定稿才能被批准。
  • Closed/Appoved 批准：批准了，可以提交改动。
  • Closed/Withdrawn 撤回：补丁被抛弃，或者不需要 CSR。
• Compatibility Kind 兼容性种类：source 编译兼容性、binary 字节码运行兼容性、behavioral 行为兼容性
• Compatibility Risk 兼容性风险等级
• Compatibility Risk Description 兼容性风险介绍：介绍具体兼容风险和为什么评某个等级
• Reviewed By 工程师批准：有了工程师批准后才能定稿提交 CSR 去被批准。
• Scope 范围：一般是 java. 模块是 SE，jdk. 模块是 JDK，对接口没影响的一般算 Implementation。
• Interface Kind 接口种类：打勾，一般核心库是 Java API。
• Fix Version/s 修复版本：必须提前选择，补丁针对的版本。没正确版本 CSR 不会审核！
• Attachments 附件：一般定义变动的 git diff 太大可以用附件上传。

兼容性种类

一般 API 修改会编译和字节码运行都会有兼容考量，但是有些特例。

Source 编译兼容

之前 Sequenced Collection 破坏编译兼容，同时实现List和Deque的类无法继续编译，reverse。但是运行时因为两个reverse字节码签名不同，老版本实现可以继续正常跑。

Binary 字节码运行兼容

比如一个方法，返回从Object变String。编译还是没问题，返回值完全兼容，但是字节码签名变了，老版本下游依赖不能用这个新方法，老方法被移除了。

Behavioral 行为兼容

比较常见。比如以前不报错现在报错，报错种类不同，返回值变化，能接收的参数种类更广这样的。

Note: This page is still under construction

java.lang.invoke, also known as JSR 292, is known for MethodHandles and invokedynamic. It is known for the support of dynamic programming languages, yet it is crucial to Java itself as time goes on. Let’s take a look at its history and its implications.

Before java.lang.invoke

We all know that the most usual way to get a MethodHandle is through MethodHandles.lookup(), which can find field accessors and methods. But didn’t reflection exist before that? Why couldn’t reflection be used?

Reflection and Unsafe

Before the appearance of invoke, reflection did exist, and this is how they were implemented:

• Method accessors used ad-hoc bytecode generation that was only removed in favor of MethodHandle in JEP 416; as of JDK 23, the infrastructure still exists to support old serialization constructor generation.
• Field accessors used Unsafe, which soon becomes notorious as a major blocker for upgrades past Java 9. Back then, it was much simpler, with only field access methods using a long offset.

So what does MethodHandle do in comparison? Each MethodHandle has a fixed MethodType; a MethodType can speed up calls significantly compared to argument conversions performed by reflection. And indeed, each invokedynamic instruction has a fixed MethodType passed to the bootstrap method.

Reading the java.lang.invoke code

Entrypoints from the VM

Since invocation happens from the VM, it would be helpful to find where the call sequences start. The entrypoints to the whole invoke system are these 3 methods in MethodHandleNatives:

• linkCallSite: Links a CallSite, i.e. an invokedynamic instruction
• linkMethod: Links a signature-polymorphic method in MethodHandle (invokeExact or invoke) or VarHandle (access methods)
• linkDynamicConstant: Resolves a CONSTANT_Dynamic to a constant value

linkCallSite and linkMethod return MemberName which points to infrastructure static methods, mostly in dynamically-generated LambdaForms (see InvokerBytecodeGenerator and Invokers too). They can also point to pregenerated bytecode, such as to VarHandleGuards methods for VarHandle, or to Invokers$Holder from pregeneration (via CDS or jlink)

Back into the VM

The execution of course comes back into JVM. The hooks are all in MethodHandle:

• invokeBasic: Used by LambdaForm code generation to easily invoke nested MethodHandles, such as ones with bound arguments (BoundMethodHandle); essentially same as invokeExact or invoke but without type conversions, as all types are “basic types” (loadable types)
• linkToVirtual, linkToStatic, linkToSpecial, linkToInterface: The most basic calls used by java.lang.invoke. Used by DirectMethodHandle.preparedLambdaForm to simulate invokevirtual, invokestatic, invokespecial, invokeinterface calls. However, they are more powerful, as they can link to hidden classes with the trailing MemberName argument while Java bytecode cannot.
  • In addition, linkToStatic is explicitly used in VarHandleGuards to invoke static methods when there are many MemberName possibilities.
• linkToNative works much like the other link methods, except it takes a trailing NativeEntryPoint. Used by NativeMethodHandle.preparedLambdaForm.

LambdaForm

Being thousands of lines long, LambdaForm is daunting to dig through. However, if you are a bytecode guru, you can check out InvokerBytecodeGenerator which converts LambdaForm to hidden classes. Also check out preparedLambdaForm in a few MethodHandle implementations. Luckily, LambdaForm is a well encapsulated class, so understanding its upstream and downstream can give you a good grasp of what it does before you dive in.

MethodType

MethodType seems simple on the surface: just a return type plus an array of parameters. What good does it do so we need it?

Turns out MethodType encapsulates some complex logic too: one is its invokers, which dictates how polymorphic methods with its type should be invoked; in addition, it is interned, just like the String for method and class names in reflection. It also has some logic for erasure to “basic types” (similar to the loadable types in bytecode) to reduce LambdaForms and code generation.

Best practices

MethodHandle and VarHandle

When using MethodHandle and VarHandle, prefer to keep them as constants (another good topic to dive into later), such as in static final fields.

Always prefer calling invokeExact; this methods is the fastest. A call to invoke, in contrast, may call asType every time when the handle’s invoked, and even if the asTypeCache doesn’t miss, since it’s a soft reference instead of a constant, it cannot be inlined.

Similarly, when declaring a VarHandle, finish the declaration with a withInvokeExactBehavior. Otherwise, the VarHandle will suffer from similar performance penalties if called with a suboptimal type (JDK-8160821).

Dynamic constants

Compared to invokedynamic bootstrap methods scattered across many classes (LambdaMetafactory, StringConcatFactory), the ConstantBootstraps method provide a lot of bootstrap methods for general-purpose dynamic constants otherwise not representable in the constant pool, such as nullConstant, primitiveClass, for use in bootstrap method arguments. There are two useful ones, getStaticFinal and invoke, which can translate otherwise eagerly initialized static final fields in a class to a lazy constant to reduce class initialization cost.

Hidden classes

Hidden classes began with Unsafe.defineAnonymousClass, which defined “VM anonymous classes”; they indeed began with invoke, as they were first used for LambdaForm implementations. Now, they have been promoted to a standalone Hidden Classes feature usable by all Java programs.

NestMates

From JEP 181:

The notion of a common access control context arises in other places as well, such as the host class mechanism in Unsafe.defineAnonymousClass(), where a dynamically loaded class can use the access control context of a host. A formal notion of nest membership would put this mechanism on firmer ground (but actually providing a supported replacement for defineAnonymousClass() would be a separate effort.)

How unexpected! Nestmates come from VM anonymous classes. Indeed, in current invoke, the generated LambdaForm$ hidden classes still have LambdaForm as their host class, though they are not nestmates.

An anecdote about nest is that they were created to enable generic specialization by subclassing in project Valhalla. (Treat this message with doubt, since I forgot about the source)

The nests also greatly simplified some Java design patterns. For example, before nests:

private static class Holder {
    static final Object instance;
}

The field declaration avoided private because java compiler has to generate accessor to access the instance; it had always generated bridge methdos to access private members in enclosing and inner classes, because these concepts don’t exist in the JVM (only packages exist).

Another anecdote is that a MethodHandle can be created for a nested enum constructor and can be called without any problem, while doing so is prohibited by reflection.

ClassData

Class data is any object passed to MethodHandles$Lookup.defineHiddenClassWithClassData(). Compared to passing the data elsewhere such as via ThreadLocal, using class data is more thread safe and less costly.

Since there are hidden classes, class data becomes necessary, as not all MethodHandle instances are representable by bytecode instructions. LambdaForm classes use class data to represent other hidden classes and MemberName for hidden class members.

Class data is usually accessed in generated code with MethodHandles.classData. It’s intentionally compatible as a bootstrap method to facilitate usage as a dynamic constant and using that constant as opposed to calling this method on each site. (Note that InvokerBytecodeGenerator does not use condy, as LambdaForm has to be ready before condy is available for use, so it stores the values in static final fields instead)

There’s an additional MethodHandles.classDataAt, but calling List.get(int)Object is preferable in actual bytecode to prevent spamming up the constant pool; classDataAt is mostly for supplying bootstrap method arguments.

Other attributes

Other important attributes of hidden classes include:

• Omission in stack traces by default
• Not modifiable by instrumentation
• Final fields are automatically “trusted” (part of constants)
• Class no longer discoverable by Class.forName

These pose risks for migration of regular generated classes to hidden classes.

Impact of java.lang.invoke

invokedynamic

Initially created to allow dynamic programming languages to better resolve calls (like Gradle’s closures), indy is also noted for its ability to provide distinct implementations on different VMs; just like library methods that evolve over time, older code using indy will use the modern code shape provided by indy, enjoying improved performance.

For example, LambdaMetafactory can try using shared-class approach (storing MemberName or MethodHandle in final fields and create a class only if the interface differs) to reduce class loading pressure when a few interfaces have a lot of implementations. Already in action is ObjectMethods where record’s object methods are being improved over time, and StringConcatFactory that relays back to StringBuilder if the concatenation is too complex.

Reflection

We have discussed how reflection was before invoke - ad-hoc classes generated for each different method. This creates a lot of classes. In comparison, JEP 416 creates a MethodHandle that may use shared LambdaForm if possible; this change might explain the slowdown observed with reflection for non-constant field/method objects. Yet it’s a good tradeoff, as it significantly reduces classloading pressure.