Replies: 12 comments 27 replies
-
@brucelwl Before drawing firm conclusions, I would increase warmup and test iterations to make sure this is not just measuring initialization: Jackson 3.x does more initial processing I think, so initialization would take longer (although even then 4x would be bit unexpected). Generally I use runtime units of not iterations but seconds of runtime: warmup needs to be typically at least 3 seconds or so (I use 5), test time multiple seconds as well. Assuming this is corrected as necessary.... It would be good to know:
Benchmarks I have been using (https://github.com/FasterXML/jackson-benchmarks/) show 3.x results to be similar to 2.x although not better (with JDK 17), but not substantially worse in most cases PS. Also: I would first remove multi-threading for tests (threads=15) when isolating numbers, esp. for small number of iterations. This to get more stable baseline -- and only then try out higher thread counts to see if that matters. |
Beta Was this translation helpful? Give feedback.
-
@cowtowncoder This is UserInfo code, a very simple entity class, By the way, I would like to clarify that I am using Openjdk 25 import java.util.ArrayList;
import java.util.List;
public class UserInfo {
String username;
List<String> tags = new ArrayList<>();
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public List<String> getTags() {
return tags;
}
public void setTags(List<String> tags) {
this.tags = tags;
}
} |
Beta Was this translation helpful? Give feedback.
-
@cowtowncoder Jackson3 uses MethodHandle, which theoretically performs much faster than Jackson2 using reflection, but even if the performance is similar, it can still be difficult to get excited |
Beta Was this translation helpful? Give feedback.
-
Here's my attempt to spruce the benchmark up a bit: @State(Scope.Benchmark)
@BenchmarkMode(Mode.Throughput)
@Measurement(time = 20)
@Warmup(time = 20)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Fork(2)
public class JacksonBenchmark {
private com.fasterxml.jackson.databind.ObjectMapper jackson2;
private com.fasterxml.jackson.databind.ObjectWriter jackson2Writer;
private com.fasterxml.jackson.databind.ObjectReader jackson2Reader;
private tools.jackson.databind.ObjectMapper jackson3;
private tools.jackson.databind.ObjectWriter jackson3Writer;
private tools.jackson.databind.ObjectReader jackson3Reader;
private UserInfo exampleUser;
private String exampleUserJson;
@Setup
public void setup() throws Exception {
jackson2 = new com.fasterxml.jackson.databind.ObjectMapper();
jackson2Writer = jackson2.writerFor(UserInfo.class);
jackson2Reader = jackson2.readerFor(UserInfo.class);
jackson3 = new tools.jackson.databind.ObjectMapper();
jackson3Writer = jackson3.writerFor(UserInfo.class);
jackson3Reader = jackson3.readerFor(UserInfo.class);
exampleUser = new UserInfo();
exampleUser.setUsername("aaa");
exampleUser.setTags(List.of("aaa", "bbb", "中文你好"));
exampleUserJson = jackson2Ser();
}
@Benchmark
public String jackson2Ser() throws JsonProcessingException {
return jackson2Writer.writeValueAsString(exampleUser);
}
@Benchmark
public UserInfo jackson2Deser() throws IOException {
return jackson2Reader.readValue(exampleUserJson);
}
@Benchmark
public String jackson3Ser() throws JsonProcessingException {
return jackson3Writer.writeValueAsString(exampleUser);
}
@Benchmark
public UserInfo jackson3Deser() throws IOException {
return jackson3Reader.readValue(exampleUserJson);
}
public static class UserInfo {
private String username;
private List<String> tags = Collections.emptyList();
public String getUsername() {
return username;
}
public void setUsername(final String username) {
this.username = username;
}
public List<String> getTags() {
return tags;
}
public void setTags(final List<String> tags) {
this.tags = tags;
}
}
} With this code, I observe a minor, but measurable, slowdown. But it is more like <10% than 5x.
|
Beta Was this translation helpful? Give feedback.
-
Ok: I re-ran JSON vanilla (no Afterburner) POJO read/write tests from https://github.com/FasterXML/jackson-benchmarks/ for combinations of:
and differences I see are minor. TL;DNR:
... almost within margin-of-error range. So this test does not reproduce slowdown. Test invocations (4/3 second warmup, 25 second total test time (5 x 5):
Raw results:
|
Beta Was this translation helpful? Give feedback.
-
@stevenschlansker Your numbers align quite well with mine, which is encouraging. @brucelwl Both Steven's and my testing suggest original results might be artifacts due to something in the testup, not Jackson or JDK versions. It is still disappointing that earlier improvements I definitely saw ~5 years ago with JDK 8 are not present any more with later versions. Although test case presented here is sort of tiny: only 2 named properties, only String values, very small payload.
|
Beta Was this translation helpful? Give feedback.
-
I used For the serializer case, Usual caveats of "I barely know what I am doing here" apply :) |
Beta Was this translation helpful? Give feedback.
-
For the deserializer case, the biggest difference I see is in the "fail on trailing tokens" check - in jackson 3, we spend about 8% of our time doing this, but it is off by default in jackson 2. Also note @cowtowncoder I think there is a merge error in javadoc in DeserializationFeature.java around line 240 |
Beta Was this translation helpful? Give feedback.
-
jackson-3-ser.html |
Beta Was this translation helpful? Give feedback.
-
I made a new attempt. I updated the test setup to have feature parity (thread local recycler, disable fail on trailing tokens)
New results:
Now, I see Jackson 3 is within margin of error for deserialize, and maybe a hair faster for serialize, for this given case of very small bean with utf-16 strings. |
Beta Was this translation helpful? Give feedback.
-
Bonus fun round, should you use
With Jackson 3, it seems marginally faster to use |
Beta Was this translation helpful? Give feedback.
-
@stevenschlansker On question of "what if we just didn't recycle", I tweaked But. On deserialization I saw no difference (!) between any pools, for 1-thread case. That suggests either:
Then I tried with threads=4, and this time there are differences; nothing drastic, all impls similar except that now (and esp with threads=16) bounded-pool is quite a bit slower. So I should probably look into where recycling itself is not occurring somehow, for read case. :-/ It'd be good to actually test recycling aspects in jackson-core and/or jackson-databind unit tests -- would require some advance mocking I suspect. Or maybe just custom-for-tests recycling pool implementation... |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I conducted benchmark tests on Jackson3 and Jackson2 through JMH, but found that the performance of Jackson3 was much worse than that of Jackson2, which did not meet my expectations for Jackson3. I don't know if my test cases were unreasonable or if there are other reasons. Do you have a stress test report for Jackson3? Here is my test code
Beta Was this translation helpful? Give feedback.
All reactions