Comparison of MATLAB Versions for CLI and GUI Performance

Although the MATLAB core language essentially has stayed the same throughout the years, the MATLAB runtime has with every release been extended with more functionality as well as featured significant improvements to the interpreter, Just-In-Time (JIT) compilation engine, and graphics capabilities. It is therefore easy to assume that the latest version always is the best to use, but as shall be seen in the following, that isn’t necessarily the case for all situations.

The FEATool Multiphysics CAE and physics simulation toolbox has been specifically designed to be as backwards compatible as possible and will work with any MATLAB version back to 2009b. Moreover, FEATool includes a fully integrated graphical user interface (GUI) as well as supports running physics and simulation models as MATLAB m-file scripts. As each click and operation using the GUI can be 1-to-1 mapped to a unique FEATool function call, modeling sessions can conveniently be recorded, saved, and exported in fully equivalent binary format, MATLAB m-file script format, or a automated GUI playback file.

With this in mind, FEATool Multiphysics (v1.12.3) is used to test and compare MATLAB performance for both CLI and GUI interfaces for several MATLAB releases. FEATool features a large number of pre-defined and built-in validation test cases and fully automated (GUI) tutorials which can be used to automatically and consistently measure performance. Timings and tests are performed on a typical modern laptop system with the Microsoft Windows 10 operating system and Intel CPU.

MATLAB Code and CLI Performance

To compare MATLAB code and command line interface (CLI) performance the hundred or so FEATool m-file script simulation models are run for each MATLAB version. These tests include everything from CAD geometry generation, PDE model set-up and pre-processing, mesh generation, linear equation solving and post-processing. The averaged total time for several runs are shown in the table below. (The tests can be run independently by starting FEATool with featool testm, which runs all the available script models in the simulation models and examples directory).

As the FEATool code base is both highly vectorized and optimized, and since Finite Element Analysis (FEA) simulations makes significant use of pre-compiled numerical libraries and sparse linear solvers such as BLAS, LAPACK, MUMPS, Umfpack/SuiteSparse etc. which are not affected by JIT improvements, a quite modest 10% speedup is seen for version 2019a compared to 2011b/2012b (significant JIT improvements were introduced with release 2015b).

MATLAB GUI Performance

In the second step, the GUI model test suite is used for timing runs. This test suite runs about 50 pre-defined multi-physics simulation models (the test suite can be run by starting FEATool with the featool testt command). The examples are completely automated so that all GUI callbacks are automatically and consistently run just as a user typically would do manually. The timing averages for the total time from several runs can be seen in the table below.

From the results it is clear that even with the improved JIT optimization that the newer MATLAB versions have, earlier MATLAB versions runs significantly faster (around 25-30% faster) than versions R2014b or later which is likely due to the newer graphics engine (handle graphics hg2) which was introduced at that point.

Note that the UI framework used by FEATool has been specifically developed and written to run fast as possible using simple procedural code, as the Mathworks GUI Layout Toolbox was also tested in development and at the time shown to have significant overhead and cost due to the object oriented design, and just not performant enough for moderately complex GUIs such as used by FEATool.

MATLAB Disk Space Requirements

Although maybe not directly related to performance it is also interesting to see how much disk space is required by the MATLAB installations, and specifically how much is has grown over the years. While earlier versions required 600-700 MB, MATLAB now requires about 2.7 GB or more (for the basic/standard MATLAB runtime not including any additional toolboxes).

Summary

By using the automated physics and simulation models available with FEATool Multiphysics we have been able to both test and see how the current MATLAB JIT optimizations (available with MATLAB R2015b or later) can result in a modest speed increase of about 10% compared to earlier versions over a span of 10 years. However, older MATLAB versions are still significantly more responsive when it compares to using GUI and visualization functionality. To MATLAB users one can therefore generally recommend using earlier versions (2011b or 2012a) when using or writing GUI applications, and newer versions for CLI applications and running m-file code scripts.

For FEATool Multiphysics users it can be convenient to have both an older and newer version of MATLAB installed simultaneously. As FEATool supports saving and exporting models to standard MATLAB m-script files from the GUI, one can work in the GUI using and older and faster release, and subsequent simulations can be exported and run on the command line in a newer version of MATLAB if compute speed is critical. Also if a larger study involving many simulation runs is to be carried out it is recommended to use a batch script to run MATLAB scripts in parallel.