Tool Integration in the IBWS
There are two aspects to tools integration in the IBWS. The first concerns shared data, and the second concerns shared components and the user experience. It is possible to have a set of completely independent tools but this leads to user frustration, inefficiency and errors in data processing. We can then think of a series of integration goals in increasing order of complexity. We need to find the right stopping point which enhances the user experience, maximizes efficiency of development, maintenance and use, and which minimizes errors in data processing.
A common Data Management System
All tools should access the same database (or system of databases) and exchange data and results via this system. For example the GDMS needs to retrieve sets of germplasm from the list manager data tables and germplasm information from the genealogy management system. Optimas should obtain pedigree information from the genealogy management system, Breeding View should read datasets from the data management system and write results back there for the next tool to access.
This level of tool integration is critical for reducing the frustration and risk of tool dependent data processing. It should be facilitated by the data access middleware and API being developed by Efficio, but to make it work tool developers have to commit to using that API, even if they do so as an alternative data access approach.
A common Workbench
The second level of tool integration is to have all the tools in the system accessible from a single platform - the IB Workbench. The minimum requirement here is that the tools share a configuration system, such as a common ini file, so that a single set-up configures the full set of tools at a common level (which databases are accessed) and at a particular level (which tool dependent defaults are set).
While the ini file system has worked well in the past, new methods involving a database component to store configuration details may be better. However we need to agree on that system, and agree to use that system for tool configuration.
The workbench must have a customizable 'menu' system to allow tools to be plugged in. This must be simple and achievable without programming. An example is the Launcher.txt file of the ICIS system. Whatever system is adopted , it needs to be described and accessible to tools developers if they are to plug their tools into the workbench.
We are promoting the IB Workflow System so tools must be 'piped' into workflows. To be effective these workflows must be customizable. Any administrator must be able to 'construct' a workflow to suit the needs of a particular project, and this construction should not involve any programming. We need to agree on how workflows are constructed and stored in the CWS.
The first consequence of this requirement is that it must be possible to deploy tools independently. In other words a constructor of a workflow may want a germplasm search at point a, a list management tool at point b, a fieldbook at point c, a cross management tool at point d, and another constructor may want the same tools in a different order. This cannot be done if half those tools reside in one application and the other half in another. Tool developers have to commit to providing independent tools even if they want to assemble them into their own applications.
A possible exception to this is in the Analytical Pipeline where Analysis Workflows are being constructed on top of statistical packages. What would be nice is to treat each Analysis Workflow as a tool, and be able to plug in a particular analysis and a certain point of the breeding workflow. (Analysis workflows should also be customizable if possible).
Many tools in the CWS share functionality. For example most require a germplasm search function, most require some list management functions, many require study and data set manipulation. The most efficient way to serve this requirement would be to have a set of applets which could be called by tools developers in the same way as data access is facilitated by sharing a common data access API. This however requires tool developers to use common development platforms and so far this has proved impossible. Everyone has good reasons to choose the development platform they have chosen, but to obtain the benefits of shared components and common 'look and feel' compromises on choice of development platform are essential. Otherwise the best we can do is provide design specifications for the common applets and have each developer develop their own versions which will soon diverge in a maintenance tangle. Are we willing to bite the bullet on this issue?
The holy grail of tool integration is to have dynamic interoperability of tools. For example a genotype viewer which shows a graphical view of the genotypes selected in a list manger. This requires agreeing and adopting a dynamic data sharing layer (above the database) and as far as I know this has never been achieved by a dispersed group of developers in the past. Are we up for it?