Nội dung

(2020) 21:456 Deschamps and Ries BMC Bioinformatics https://doi.org/10.1186/s12859-020-03727-8 SOFTWARE Open Access EMU: reconfigurable graphical user interfaces for Micro-Manager Joran Deschamps* *Correspondence: joran.deschamps@embl.de European Molecular Biology Laboratory, Meyerhofstrasse 1, Heidelberg, Germany and Jonas Ries Abstract Background: Advanced light microscopy methods are key to many biological studies. Their ease of use depends, besides experimental aspects, on intuitive graphical user interfaces (GUI). The open-source software Micro-Manager offers a universal GUI for microscope control but requires implementing plugins to further tailor it to specific systems. However, GUIs are often tailored to a single system. Since even similar devices can have different Micro-Manager device properties, such as power percentage versus absolute power, directly transferring a GUI to another instrument usually requires changing the source-code. Results: We developed Easier Micro-Manager User interface (EMU), a Micro-Manager plugin, to simplify building flexible and reconfigurable GUIs. EMU can be seamlessly used with the Java Swing library to create device-independent GUIs for Micro-Manager. Such GUIs are easily transferred to another microscope thanks to an intuitive configuration menu that includes mapping of the device properties to the GUI functionalities and customization of the graphical elements. We also provide resources such as user and programming guides, a tutorial and code examples. Conclusions: Micro-Manager users now have a powerful tool to improve the user experience on their instruments. EMU GUIs can be easily configured for new microscopes and shared with other research groups. In the future, newly developed GUIs will be added to EMU to benefit the whole community. Keywords: Microscopy, User interface, GUI, Micro-Manager Background Light microscopy is an ever-growing field with countless applications in biosciences. A substantial portion of technology developments and cutting-edge research are carried out on custom microscopes because of their high flexibility. Beyond the mechanical and optical requirements of such microscopes, researchers face the challenges of controlling the hardware and presenting the users with an intuitive graphical user interface (GUI). While control software are often developed for a specific instrument, a number of more general © The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Deschamps and Ries BMC Bioinformatics (2020) 21:456 platforms are available. These software can be commercially available, such as MetaMorph (Molecular Devices), SlideBook (3i), SciScan (Scientifica) or ScanImage (Vidrio) [1], or shared online by their authors. They span a variety of programming languages, allowing users to work with their preferred tools and facilitating post-processing and analysis. Examples of such microscope control software are found for LabView (National Instruments) [2], Matlab (MathWorks) [3] or Python [4–7]. Another well-established alternative is Micro-Manager (μManager) [8], an open-source software written in C++ and Java, and based on ImageJ [9]. μManager is a ready-to-use platform compatible with a wide range of hardware devices. The support for new devices is community driven and constantly expanding. In addition, μManager features the possibility to run scripts or plugins to perform custom experiments. μManager offers a universal GUI and some customization tools that are easy to set-up and use. However, because they are aimed at covering general needs, they cannot rival with a tailored GUI in terms of user experience. The preferred way to implement a GUI in μManager is by writing a plugin in Java, which is automatically detected at start-up and can be loaded from the main menu. Several dedicated Java libraries exist in order to build a GUI, among which the widely used Swing toolkit [10]. In order to simplify the programming task, most major Java integrated development environments provide graphical tools based on Swing. With only a basic understanding of Swing, developers can assemble complex GUIs by placing (“drag-and-drop”) components on a panel or a frame. In order to control a microscope, the Swing GUIs should be made compatible with μManager plugin system and call its application programming interface (API) to modify the devices’ states whenever users interact with the graphical components. Yet, similar devices might have different properties, such as absolute laser power versus laser power percentage; as well as different property state values, such as “On” versus “1”. These differences originate from the absence of industry standards for device communication API or simply from distinct inherent working principles. Additionally, device properties are defined in μManager device adapters, which are implemented by the manufacturers or individual researchers, and lack device property standardization as well. Therefore, in order to benefit the community, μManager GUIs should be flexible enough to accommodate these discrepancies without requiring modifying the source code. Several GUIs have been developed to control complex microscopes with μManager, such as for the Olympus IX83 [11], OpenSPIM [12], OpenSpin [13] and diSPIM [14] microscopes. Because these GUIs perform specialised acquisitions and hardware control, they are meant to work with very specific hardware devices. As a result, any departure from the original devices requires modifying the source-code to achieve compatibility. Regrettably, this is a common trait of GUIs, which are generally bound to a single system and never benefit other users. Here, we present a framework, called Easier Micro-Manager User interfaces (EMU), allowing developers to build μManager GUIs without explicit references to the devices or their properties. While EMU GUIs are, as previously, assembled using Swing, they additionally declare settings, properties and parameters. These are automatically aggregated upon starting EMU and can be configured through an intuitive graphical interface. The EMU configuration specifies the mapping between device and GUI properties, as well as the settings’ and parameters’ values. Thus, each EMU GUI is independent from the devices and can be rapidly configured to suit a microscope or accommodate device Page 2 of 13 Deschamps and Ries BMC Bioinformatics (2020) 21:456 Page 3 of 13 Fig. 1 EMU plugin example. SimpleUI is a GUI available by default in EMU. It can control four lasers and an optional filterwheel. The solid box delimits the ConfigurableMainFrame instance while the dashed boxes are the two base ConfigurablePanel objects of the GUI. All titles and colors are GUI parameters, while the GUI properties are, for each laser, the power percentage and on/off, and the filter wheel position. Additional GUI parameters allow disabling the on/off buttons. Finally, a plugin setting hides or shows the filter wheel panel exchanges. Finally, EMU aims at offering a repository of already existing GUIs for the community to use. Implementation EMU is a μManager 2.0.0-gamma plugin. It is written in Java and includes its own plugin system. EMU can be started from the plugin menu of μManager. When starting EMU for the first time, users can choose a GUI among the list of available EMU plugins, before configuring it using the EMU configuration menu. The configuration file is saved locally and is loaded automatically at the next start. In the next sections, we showcase how to implement an EMU plugin with code examples and how to configure the plugin in μManager. Implementing an EMU plugin The EMU framework is based on the Java Swing toolkit [10]. EMU plugins consist of multiple ConfigurablePanel objects arranged within a single ConfigurableMainFrame Table 1 UIProperty class and child classes. A UIProperty object only changes the state of a device property within the μManager device property limits or allowed values. UIProperty child classes have additional constraints, as specified in the second column. In the EMU configuration menu, these constraints lead to additional fields as indicated in the third column Class Note UIProperty General GUI property in EMU configuration menu device and property drop-down lists SingleStateUIProperty Accepts a single-state + field for the state value TwoStateUIProperty Accepts an On and an Off state + fields for the On and Off values MultiStateUIProperty Accepts a fixed number of states + field for each state value RescaledUIProperty Rescales value to slope*v+offset + fields for the slope/offset values Deschamps and Ries BMC Bioinformatics (2020) 21:456 instance, as illustrated in Fig. 1. ConfigurablePanel and ConfigurableMainFrame are subclasses of the Swing classes JPanel and JFrame, respectively. A ConfigurablePanel is a unit of device control. In the following code snippet, we create a subclass of ConfigurablePanel similar to the laser panel of Fig. 1, with a slider to set the laser power and a toggle button to turn it on or off. We also create a border, with title, around the graphical components. Since ConfigurablePanel is a subclass of JPanel, the laying out of components is performed using Swing: Several abstract methods from ConfigurablePanel must additionally be implemented (see Additional file 1). Three of these methods concern the GUI properties: initilizeProperties, addComponentListeners and propertyhasChanged. In initilizeProperties, we need to declare the GUI properties. Since we want to control two device properties, laser power and laser on/off, we have to declare two GUI properties. Different UIProperty subclasses exist (see Table 1 and Additional file 2), and the choice of the class depends mainly on the graphical components to which it is linked. For instance, the slider allows setting a value between 0 and 100 by default. As we do not know whether the device property ultimately linked to the slider will be a power percentage or an absolute power, we should declare a RescaledUIProperty. In the EMU configuration menu, this type of property allows users to set scaling factors mapping the slider value to a suitable range with respects to the device property (see Fig. 2). Likewise, the laser operation is controlled by a toggle button. Therefore we choose a property with only two states (TwoStateUIProperty). The on and off states are set in the EMU configuration menu as well. Declaration of the GUI properties is done as follows: Page 4 of 13 Deschamps and Ries BMC Bioinformatics (2020) 21:456 When the user interacts with a graphical component, the state of the corresponding device property should change. In Swing, you can register a listener with a component, the former being called when the component’s state is modified. In a listener implementation, the ConfigurablePanel should call setPropertyValue(String, String) (see Additional file 1) to update the GUI property given the component’s new state. In turn, the UIProperty will change the state of the device property. The listeners can be registered with the components in the constructor or in the addComponentListeners method. Alternatively, the EMU SwingUIListeners class provides a number of static methods that cover common cases. For instance, here, a convenient way to have the slider and toggle button modify their respective properties is: Whenever the state of the graphical components needs to be updated, the ConfigurablePanel instances are notified, which triggers the propertyhasChanged method. In Fig. 2 Configuring GUI properties. In the “Properties” tab of the EMU configuration menu, users can map device properties to GUI properties using drop-down lists. Additionally, some GUI properties have state values than need to be specified. Here shown for SimpleUI Page 5 of 13 Deschamps and Ries BMC Bioinformatics (2020) 21:456 this method, we need to update the state of the graphical components to reflect the device properties’ values. Here, this means setting the slider value to the (rescaled) laser power, or switching the toggle button to the state of the laser operation. This is done in the following way: Note that here, we made use of a TwoStateUIProperty class method (see Additional file 2). With these three methods implemented, any numerical device property can be linked to the slider, while any device property can be switched between two states by the toggle button. Another aspect of EMU panels are the GUI parameters, which are meant to add additional levels of customization to the panel, such as title, colors or button texts. Similarly to the properties, they are declared in initializeParameters. Parameters are not modified by the panel and their value only changes at start-up when the configuration is loaded, or when a new configuration is saved, which triggers a call to the parameterhasChanged method. Various types of parameters are available in EMU (see Table 2 and Additional file 3). Since we added a titled border to the panel, we can use a StringUIParameter to let users choose the border title: Other mechanisms make EMU panels flexible but are beyond the scope of this section, for instance: internal properties (see Additional file 4), which are values shared between panels; the possibility to map multiple GUI properties to the same device property; or the mapping of a GUI property to a bundle of device properties, known as “configuration group preset” in μManager. Page 6 of 13 Deschamps and Ries BMC Bioinformatics (2020) 21:456 Page 7 of 13 Table 2 UIParameter child classes. Each UIParameter child class holds a member variable represented by the type indicated in the second column. In the EMU configuration menu, the GUI parameters appear as specified in the third column Class Parameter type in EMU configuration menu BoolUIParameter Boolean checkbox ColorUIParameter java.awt.Color drop-down list of colors ComboUIParameter String drop-down list of Strings DoubleUIParameter Double field IntegerUIParameter Integer field StringUIParameter String field UIPropertyParameter String drop-down list of UIProperty labels Finally, the ConfigurablePanel subclasses are assembled within a single ConfigurableMainFrame subclass. The latter declares its own parameters, called settings (see Table 3 and Additional file 5), which are instantiated in getDefaultPluginSettings. In the same vein as GUI properties and parameters, their values are defined in the EMU configuration. ConfigurablePanel subclass instances should be created in the initComponents methods, in order to be able to retrieve the settings’ values at runtime. In the following example, the ConfigurableMainFrame subclass uses an IntSetting to let users choose the number of lasers in the GUI. In order to be detected by EMU, one additional class is required (UIPlugin) before packaging the GUI into a .jar file. The detailed steps can be found in the EMU guide [15], Deschamps and Ries BMC Bioinformatics (2020) 21:456 Page 8 of 13 Table 3 Setting child classes. Each class holds a member variable represented by the type indicated in the second column. All Setting child classes appear in the EMU configuration menu as a field, except for the BoolSetting settings, which appear as checkboxes Class Setting type in EMU configuration menu BoolSetting Boolean checkbox DoubleSetting Double field IntSetting Integer field StringSetting String field including the source code for the plugin of Fig. 1. In the next section, we will look at a the configuration of the latter. Configuring an EMU plugin The EMU configuration menu is shown each time a plugin without known calibration is started. Later, it can be accessed by clicking on the menu bar “Configuration” (see Fig. 1), then “Modify configuration”. The EMU configuration menu consists of four tabs: “Plugin Settings”, “Properties”, “Parameters” and “Global Settings”. In the “Properties” tab (see Fig. 2 for SimpleUI), users can map μManager device properties to GUI properties by first selecting the device in the drop-down list of the second column, then the relevant device property in the third column. Some GUI properties have additional states that need to be configured as well (see Table 1). In particular, this is the case for the GUI properties used in the previous section. States are related to the actual device property values and can be easily inferred from the “device property browser”. The latter is accessible from the “Devices” menu in μManager main window. In the “Parameters” tab (see Fig. 3), each GUI parameter appears as a field, a drop-down list or a checkbox, depending of their type (see Table 2). Finally, the other tabs are the frame settings (“Plugin Settings”) or EMU options (“Global Settings”). After saving the EMU configuration, a human-readable file is automatically created in a subfolder of the μManager installation folder. Multiple configurations can coexist and be saved in the EMU configuration file. The EMU menu (see the menu bar in Fig. 1) allows users to switch between GUIs or between configurations in a single click. Finally, at each start of EMU, the configuration file is loaded and the last known plugin configured. Example cases To illustrate the flexibility of EMU plugins, we can consider a simple example such as SimpleUI (see Fig. 1). The plugin controls four lasers, allowing users to turn their emission on and off and change their power percentage. Laser names and colors can be set in the EMU configuration menu (see Fig. 3). Several cases can be encountered when working with lasers in μManager: (i) lasers have an operation (on/off ) and a power percentage device property, (ii) some lasers have an absolute power instead of a power percentage device property and (iii) some lasers do not have a laser emission device property [16]. In (i), the GUI properties can be mapped to the devices and their relevant device properties. Since the GUI property representing the laser percentage is an instance of RescaledUIProperty (see Table 1), users can leave the slope and offset values (see Fig. 2) equal to the default 1 and 0, respectively. For the lasers in (ii), the laser percentage GUI property is mapped to an absolute power device property. There, the slope parameter should be set to max/100 (with max being the maximum laser power), effectively rescaling the power Deschamps and Ries BMC Bioinformatics (2020) 21:456 Fig. 3 Configuring GUI parameters. In the “Parameters” tab of the EMU configuration menu, users can set the values of the GUI parameters. Here shown for SimpleUI percentage to the range {0, max} of the device property. Finally, for lasers in the case (iii), no laser operation device property exists. The laser operation GUI properties should remain unconfigured. Using the GUI parameters, the on/off buttons can be disabled. In all three cases, the GUI can be configured to reflect an accurate picture of the microscope lasers in spite of their differences. The previous example plugin is simple. However there is no limit on how complex an EMU plugin can be. We routinely use htSMLM [17] (see Additional file 6), an EMU plugin aimed at controlling a wide-field microscope for localization microscopy [18–20]. htSMLM features four generic lasers, two optional lasers compatible with the iBeamSmart series from Toptica, an axial focusing panel, up to four filter wheels and multiple toggle buttons. It also includes tools to automate activation in localization microscopy and to perform acquisition series (localization microscopy, multi-slice localization, snapshot, time series, z-stack) that take into account the device properties linked to the GUI. htSMLM has been used by scientists with a wide variety of backgrounds and projects, ranging from biology to optics [21–24]. Discussion μManager GUIs are too often tailored to a specific system and are not usable for instruments with similar, yet different, devices. Therefore, such GUIs are only shared when the Page 9 of 13 Deschamps and Ries BMC Bioinformatics (2020) 21:456 system has the potential to be broadly replicated. Users without programming skills are limited to the μManager main GUI window for device control. While the main GUI covers general need and is thus applicable to most instruments, user experience can be largely improved by using more specialized GUIs in conjunction with the main window. EMU fills this gap by providing device-independent and easily configurable GUIs for μManager. Since microscopes are composed of a wide variety of devices, the diversity and redundancy of devices prevent automated allocation of the device properties to graphical components without using stringent constraints. In order to give a high degree of flexibility to developers, EMU consists of a set of classes that can be seamlessly inserted in a Swing GUI. The two main components, frames and panels, declare parameters matching the constraints of the graphical components rather than those of the intended devices. Hence, assumptions are limited to whether a certain type of device can be controlled by a specific graphical component. For instance, laser power is a device property that is undoubtedly determined by a number within a range and can therefore be represented graphically by a slider linked to a RescaledUIProperty. Similarly, a filter wheel has fixed positions, regardless of whether the actual device positioning is continuous or discrete, and can be paired with a set of buttons linked to a MultiStateUIProperty. The implementation and graphical layout of an EMU GUI is left to developers. Because EMU provides an advanced configuration system and the mechanism of interaction with the device properties, they can focus solely on the GUI design and the choice of properties and parameters fitting the GUI desired functionalities. EMU automatically aggregates the GUI properties and parameters in its configuration menu, which makes the framework particularly beneficial for complex GUIs. Moreover, EMU is compatible with “drag-anddrop” software found in most integrated development environment. Since EMU’s API is designed to be simple and involves only a few additional lines of code, plugins can be built and put to use rapidly. The goal of EMU is to promote GUIs that are transferable between similar instruments. Indeed, a GUI taking advantage of EMU’s flexibility will be usable on any other microscope consisting of similar devices. Nonetheless, a few limitations can be encountered. If the GUI was designed to work with a device property specific to a single manufacturer, then the corresponding functionality will not be of interest to other users. In such a case, developers should ensure that settings or parameters can be used to simply disable this functionality. As an example, htSMLM was designed to control, among others, iBeamSmart lasers (Toptica). Since these lasers have unconventional properties, such as the so-called fine settings, the corresponding panels can be disabled using parameters. Note that unused properties do not alter the function of the rest of the GUI. Similarly, limitations stemming from the plugin design itself can occur. For instance, a maximum laser power device property expected to set the upper bound of a slider is obviously not compatible with lasers that do not have such a property. Using a parameter to set the maximum value is an easy work-around for developers. Alternatively, the slider could represent a percentage, rather than an absolute value, and be linked to a RescaledUIProperty. Finally, unconventional device property implementations can potentially be found in μManager, such as numerical device properties implemented with a string type. These would prevent allocating it to RescaledUIProperty GUI properties, which can only be paired with float or integer device properties. Other GUI properties (see Table 1), on the other hand, are compatible with any device property. Page 10 of 13