Black Friday Sale

Flutter Foundations Course

Buy now and get 33% off the regular price!

View Course

How to Parse Large JSON Data with Isolates in Dart 2.15

Source code on Dartpad

Parsing JSON data is a very common task for many apps.

If our JSON payload is small, it's safe to run the parsing code on the main isolate (which is where our app UI runs).

But if our JSON payload is large, parsing it may take longer than the frame budget (16 ms for a 60Hz refresh rate) and we could experience dropped frames and UI jank.

In this scenario, it's best to run all the parsing code in a worker isolate that runs concurrently to the main isolate without affecting the UI.

So in this article, we'll learn how to parse JSON in two ways:

  • using the compute() function
  • by spawning an isolate to do the work and calling Isolate.exit() when done

We will focus primarily on the code to parse JSON in Dart, but I will also share an example Flutter app so that we can see how everything fits together. If you're curious, here it is running on Dartpad:

JSON Parsing with isolates on Dartpad
JSON Parsing with isolates. View on Dartpad.

How to parse JSON data in Dart

Our goal is to parse a JSON file that contains a list of all the tutorials on my site. Here's a snapshot for reference:

{ "results": [ { "title": "Flutter Tutorial: Stopwatch App with Custom UI and Animations", "url": "https://codewithandrea.com/videos/stopwatch-flutter-ui-tutorial/", "date": "Dec 6, 2021", "contentType": "video" }, { "title": "Responsive layouts in Flutter: Split View and Drawer Navigation", "url": "https://codewithandrea.com/articles/flutter-responsive-layouts-split-view-drawer-navigation/", "date": "Nov 26, 2021", "contentType": "article" }, { "title": "How to create a Flutter GridView with content-sized items", "url": "https://codewithandrea.com/articles/flutter-layout-grid-content-sized-items/", "date": "Nov 24, 2021", "contentType": "article" } ] }

This example only shows 3 items, but the whole file contains 140 items and weighs 28KB.

So let's start by writing a type-safe SearchResult class that we'll use to parse this data:

class SearchResult { SearchResult({ required this.title, required this.url, required this.date, }); final String title; final String url; final String date; factory SearchResult.fromJson(Map<String, dynamic> data) { return SearchResult( title: data['title'], url: data['url'], date: data['date'], ); } }

If you're not familiar with the factory constructor syntax above, check my essential guide to parsing JSON in Dart/Flutter.

Parsing JSON with isolates

Now comes the fun part.

Let's start by creating a SearchResultsParser class that can parse an encoded JSON string into a List<SearchResult>:

import 'dart:convert'; class SearchResultsParser { List<SearchResult> _decodeAndParseJson(String encodedJson) { final jsonData = jsonDecode(encodedJson); final resultsJson = jsonData['results'] as List<dynamic>; return resultsJson.map((json) => SearchResult.fromJson(json)).toList(); } }

Note how the _decodeAndParseJson() method is not asynchronous, but it can still take a long time to execute if the number of results is large.

As we said, we want to parse the encoded JSON in the background.

This is easily done by adding a new parseInBackground() method that uses the compute() function:

import 'dart:convert'; import 'package:flutter/foundation.dart'; class SearchResultsParser { Future<List<SearchResult>> parseInBackground(String encodedJson) { // compute spawns an isolate, runs a callback on that isolate, and returns a Future with the result return compute(_decodeAndParseJson, encodedJson); } List<SearchResult> _decodeAndParseJson(String encodedJson) { final jsonData = jsonDecode(encodedJson); final resultsJson = jsonData['results'] as List<dynamic>; return resultsJson.map((json) => SearchResult.fromJson(json)).toList(); } }

That's it!

By using compute() we can move our parsing code to the worker isolate and get all the performance benefits we want.

But is there more to this?

Fast Concurrency with Isolate.exit()

As it turns out, the compute() function has been updated in Flutter 2.8 (Dart 2.15) to take advantage of Isolate.exit(), a new method that can be used to pass the result back to the main isolate in constant time.

If you want to run a task in a worker isolate and return the result to the main isolate, the compute() function is all you need.

But I think it's interesting to learn how to do the same using the isolate APIs.

So here's an alternative implementation of our SearchResultsParser class:

class SearchResultsParser { // 1. pass the encoded json as a constructor argument SearchResultsParser(this.encodedJson); final String encodedJson; // 2. public method that does the parsing in the background Future<List<SearchResult>> parseInBackground() async { // create a port final p = ReceivePort(); // spawn the isolate and wait for it to complete await Isolate.spawn(_decodeAndParseJson, p.sendPort); // get and return the result data return await p.first; } // 3. json parsing Future<void> _decodeAndParseJson(SendPort p) async { // decode and parse the json final jsonData = jsonDecode(encodedJson); final resultsJson = jsonData['results'] as List<dynamic>; final results = resultsJson.map((json) => SearchResult.fromJson(json)).toList(); // return the result data via Isolate.exit() Isolate.exit(p, results); } }

Note that there are a few differences:

  • we explicitly create a new isolate with Isolate.spawn() in the parseInBackground() method and wait for it to complete
  • we decode and parse the json inside _decodeAndParseJson() and return the result via Isolate.exit()
  • the _decodeAndParseJson() method passed to Isolate.spawn() can only take one SendPort argument, so we have to access the encodedJson as an instance variable that is set by the constructor

Once again, you don't need Isolate.spawn() and Isolate.exit() for this use case, because the compute() function does the same work with less code.

How large should a JSON file be to parse it in the background?

Should it be 10KB? 100KB? 1MB? More?

While I don't have a precise answer to this, we can use a scientific approach.

That is:

  • we should run the app in profile mode on a low-end device
  • test different payload sizes and see if we get skipped frames on the main isolate

If that sounds like too much work, we can stay on the safe side and perform parsing in the background for payloads of 10KB or more. After all, this can be done with one line of code:

compute(_decodeAndParseJson, encodedJson)

Should the networking code run on the worker isolate?

So far, we have only moved the JSON parsing code to the worker isolate.

But what about the networking code? Consider this APIClient class:

import 'package:http/http.dart' as http; class APIClient { Future<List<SearchResult>> downloadAndParseJson() async { // get the data from the network final response = await http .get(Uri.parse('https://codewithandrea.com/search/search.json')); if (response.statusCode == 200) { // on success, parse the JSON in the response body final parser = SearchResultsParser(); return parser.parseInBackground(response.body); } else { // on failure, throw an exception throw Exception('Failed to load json'); } } }

This code uses the http package to fetch the data from the network, and then delegates the parsing logic to the SearchResultsParser class we have created.

But shouldn't the http.get() call also run on the worker isolate?

As it turns out, networking and file IO run on a separate task runner. And when an IO operation is done, the result is passed back to the main isolate.

In other words: we can safely perform IO using Future-based APIs, and Dart will do the right thing under the hood.

There is a fundamental difference between waiting without blocking (using async code) and running code in parallel (using isolates). This video about Async vs Isolates explains it very well.

Example Flutter App

Now that we've taken care of the JSON parsing and networking code, we can create a simple Flutter app that shows all the search results.

This uses the Riverpod package to tie everything together, but any other dependency injection package would work too.

Here it is, running on Dartpad:

JSON Parsing with isolates on Dartpad
JSON Parsing with isolates. View on Dartpad.

Conclusion

When dealing with large JSON data, we should offload the parsing code to a worker isolate and the easiest way to do this is with the compute() function.

As of Flutter 2.8, the compute() function can return the result data to the main isolate in constant time, thanks to the fast concurrency features introduced by Dart 2.15.

References

If you want to learn more about isolates, check this page in the official documentation:

There are also some very good videos from the Flutter team on this topic:

Here are the Flutter 2.8 and Dart 2.15 announcements:

And this StackOverflow thread explains how Dart manages IO operations under the hood:

Happy coding!

Want More?

Invest in yourself with my high-quality Flutter courses.

The Complete Dart Developer Guide

The Complete Dart Developer Guide

Learn Dart Programming in depth. Includes: basic to advanced topics, exercises, and projects. Fully updated to Dart 2.15.

The Complete Flutter Course Bundle

The Complete Flutter Course Bundle

Learn about State Management, App Architecture, Navigation, Testing, and much more by building a full-stack Flutter eCommerce app on iOS, Android, and web.

Flutter Animations Masterclass - Full Course

Flutter Animations Masterclass - Full Course

Master Flutter animations and build a completely custom habit tracking application.