AVRO-2335: Remove Joda Time library#631
Merged
nandorKollar merged 3 commits intoapache:masterfrom Sep 17, 2019
Merged
Conversation
782d76c to
75b7da6
Compare
75b7da6 to
8dd377a
Compare
nandorKollar
reviewed
Sep 5, 2019
| * JodaTimeConversions.TimestampMicrosConversion()); | ||
| */ | ||
| @Test | ||
| public void testDynamicSchemaWithDateConversion() throws ClassNotFoundException { |
Contributor
There was a problem hiding this comment.
I recommend adding similar test cases to TestTimeConversions class now - I can't see any ReflectData tests there. Here are the cases I'm thinking of:
@Test
public void testDynamicSchemaWithDateConversion() throws ClassNotFoundException {
Schema schema = getReflectedSchemaByName("java.time.LocalDate", new TimeConversions.DateConversion());
Assert.assertEquals("Reflected schema should be logicalType date", DATE_SCHEMA, schema);
}
@Test
public void testDynamicSchemaWithTimeConversion() throws ClassNotFoundException {
Schema schema = getReflectedSchemaByName("java.time.LocalTime", new TimeConversions.TimeMillisConversion());
Assert.assertEquals("Reflected schema should be logicalType timeMillis", TIME_MILLIS_SCHEMA, schema);
}
@Test
public void testDynamicSchemaWithTimeMicrosConversion() throws ClassNotFoundException {
Schema schema = getReflectedSchemaByName("java.time.LocalTime", new TimeConversions.TimeMicrosConversion());
Assert.assertEquals("Reflected schema should be logicalType timeMicros", TIME_MICROS_SCHEMA, schema);
}
@Test
public void testDynamicSchemaWithDateTimeConversion() throws ClassNotFoundException {
Schema schema = getReflectedSchemaByName("java.time.Instant", new TimeConversions.TimestampMillisConversion());
Assert.assertEquals("Reflected schema should be logicalType timestampMillis", TIMESTAMP_MILLIS_SCHEMA, schema);
}
@Test
public void testDynamicSchemaWithDateTimeMicrosConversion() throws ClassNotFoundException {
Schema schema = getReflectedSchemaByName("java.time.Instant", new TimeConversions.TimestampMicrosConversion());
Assert.assertEquals("Reflected schema should be logicalType timestampMicros", TIMESTAMP_MICROS_SCHEMA, schema);
}
private Schema getReflectedSchemaByName(String className, Conversion<?> conversion) throws ClassNotFoundException {
// one argument: a fully qualified class name
Class<?> cls = Class.forName(className);
// get the reflected schema for the given class
ReflectData model = new ReflectData();
model.addLogicalTypeConversion(conversion);
return model.getSchema(cls);
}
@Fokko what do you think?
Contributor
Author
There was a problem hiding this comment.
I've restored the above tests. The VCS is a bit messy because I've removed TestJodaTimeConversions.java and removed everything with Jsr310 from the filenames.
nandorKollar
approved these changes
Sep 5, 2019
Contributor
nandorKollar
left a comment
There was a problem hiding this comment.
Left a minor comment related to a test case, otherwise LGTM!
Contributor
|
+1 |
3 tasks
sijie
pushed a commit
to apache/pulsar
that referenced
this pull request
Jan 6, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
tuteng
pushed a commit
to AmateurEvents/pulsar
that referenced
this pull request
Mar 21, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior. (cherry picked from commit d6f240e)
tuteng
pushed a commit
to apache/pulsar
that referenced
this pull request
Apr 13, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior. (cherry picked from commit d6f240e)
jiazhai
pushed a commit
to jiazhai/pulsar
that referenced
this pull request
May 18, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior. (cherry picked from commit d6f240e)
huangdx0726
pushed a commit
to huangdx0726/pulsar
that referenced
this pull request
Aug 24, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
sijie
pushed a commit
to apache/pulsar-adapters
that referenced
this pull request
Nov 6, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
sijie
pushed a commit
to apache/pulsar-adapters
that referenced
this pull request
Nov 6, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
sijie
pushed a commit
to apache/pulsar-presto
that referenced
this pull request
Nov 11, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
sijie
pushed a commit
to apache/pulsar-presto
that referenced
this pull request
Nov 11, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
sijie
pushed a commit
to apache/pulsar-release
that referenced
this pull request
Nov 12, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
sijie
pushed a commit
to apache/pulsar-release
that referenced
this pull request
Nov 12, 2020
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
lhotari
pushed a commit
to apache/pulsar-sql
that referenced
this pull request
Oct 18, 2024
### Motivation Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions. https://blog.godatadriven.com/apache-avro-1-9-release ### Modifications Avro 1.9 has some major changes: - The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631) - Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283) - Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema: ```json { "name": "fieldName", "type": [ "null", "string" ], "default": "defaultValue" } ``` The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make sure you have checked all steps below.
Removes the Joda time library and makes Java JSR310 dates the default.
Jira
Tests
Commits
Documentation