As of August 1st, 2008, the Edutech web site will no longer be updated. Edutech was funded by the Swiss Virtual Campus programme, which ended on July 31st, 2008. Some activities will be taken over by the e-Learning Services group at the SWITCH foundation.

LMS Evaluation Report

hide details | show criteria descriptions

Products

IBT Server 6.1

Time4You: http://www.time4you.de

Last major evaluation update: Dec '02

strong points

  • full XML/XSL support
  • support for SCORM compliant learning modules
  • API to all important server functions (http/JavaScript and/or JSP programming interfaces) -> toolkit approach
  • the entire system (layout, functionalities) can be fully customized
  • server runs on all operating systems with Java Virtual Machine
  • good multi-language support
  • clean and modern technical design
  • modular architecture
  • runs on all modern browsers

weaknesses

  • many features are only available through programming
  • difficult to use for course designers - non-intuitive user interface
  • system cannot really be used out-of-the-box
  • running/customizing the server requires skilled and experienced staff (XML, XSL, JSP, HTML, JavaScript and IBT-architecture knowledge)
  • complex system, difficult to get used to it
  • small user base in universities

Student's Environment

Student's Environment

Ease of Use
The environment can mostly be used as a common web page. Pages can be printed and saved, and their fonts are scalable. Not all common navigation tools work as expected: pages can't easily be bookmarked (frame problem), URLs can't be copy-pasted (frame problem). However, the back button works as expected.

It should be possible for students to cope with the limitations, although the impossibility to copy-paste URLs (impossibility to send a web reference by email) is a severe restriction.

browser bookmarks
partially. Frameset is not always reconstructed
URL access possible
partially. Frameset is not always reconstructed
in-system bookmarks
available as 'navigation history'
print current page
works, if user knows how to deal with frames in web sites (i.e. select a 'current' page)
save page on local disk
yes
find word in current page
yes
no frames
the default configuration uses frames. The system is flexible enough to be represented without frames (requires customization)
font scalable
yes (with IE6 and Mozilla 1.2)
Compliant with common web technology
The environment works well with common modern web browsers like Mozilla 1.2 and IE6.
Mozilla 1.7
-
Netscape 6 ok
yes, if JavaScript is activated
IE 6 ok
yes, if JavaScript is activated
other browsers ok
-
web-tablets, set top boxes ok
-
Functional environment
A flexible and powerful search tool allows to search for keywords in content pages, discussion forums, resource metadata, message boards etc. The search engine can also be restricted to predefined areas. A context sensitive help tool is available
keyword search
yes. Full text search in content pages and discussion forums
configurable environment
the students have little possibilities to customize their environment
effective navigation aids
-
context sensitive help
available.
Collection of Tools
-

Tutoring and Didactics

Ease of use
the tools in the teletutoring workspace are fairly easy to use.
Communication
The communication tools are quite advanced. All basic asynchronous communication modes are available, as well as chat rooms.

The default layout of the communication tools (i.e. the forum) is less pleasant. Some developers might want to customize it.

forum threaded
Yes
forum searchable
Yes
private workgroup forums
Yes, as part of courses or forums
internal mailing system
yes. A powerful internal messaging system is available. Message types: mail, forum topic, pinboard item, chat item, ...
mailing system interoperable with emails
-
chat
yes, with entry/leave listeners.
whiteboard with integr. chat
available in new version
Instant Messaging
-
Students management
The student management is powerful, and allows to create groups for teamwork activities.
class enrollment by tutor
yes
class enrollment by student
tutors can build groups
yes, fully fledged group management
students can build groups
Activity tracking
All important activity tracking functions are available, and graphicl reportc can be generated.

- activity tracking ist in jeder der angesprochenen funktionen seit der v6 verfügbar. über die module ibt management information, ibt skill management oder auch den teletutoring workspace/desktop (ab v7) in je unterschiedlichen hinsichten mit standardberichten und -statistiken, die um individuelle berichte ergänzt werden können.

individual page/tool access
yes
class page/tool access
yes
individual activity in forum
yes
class activity in forum
yes
individual performance in tests
yes
class performance in tests
yes
class assignment performance
-

Course Developement

Ease of Use
The authoring environment contains a fully featured content editor (xhtml editor) and an editor to organize the structure of a course. Both editors are web based and are extensively using JavaScript and Forms. The web authoring suite also has a so called 'rapid developer' that allows to easily assemble a default course structure in just a few minutes.

Time4You suggests to use the web based xhtml editor and structure editor to create courses.

Our experience however shows, that the xhtml editor is difficult to use, and requires more knowledge about xhtml files than with desktop (x)html editors. Even on a fast network the editor reacts slowly, which is a typical drawback of complex web applications. Informal polls reveal, that many users perceive the web based xhtml editor as clumsy and hence refuse to work with it.

As far as the integration of the numerous tools into a course is concerned: everything is done over the logically/hierarchically web interface. Unfortunately, this does not mean, that it is simple to use! Example: configuring a search tool for some content pages or a discussion forum is not intuitive, and we can't expect occasional course designers to do it. For an experienced support staff member however, this is a fairly easy task.

online editor for course organization
Yes
online editor for content structure
Yes
Web interface to up/download resources
Yes
up/download of resource packages
Yes
Drag & Drop loading
-
offline editor for course structure
Yes,
offline editor for content structure
Possibility to install an IBT server locally. The prepared content chapters can be zipped and transferred to the main server.
Linking
cross-course linking possible
Flexible Development Framework
SCORM is supported, and this allows to develop complex content modules that interact with the server over the runtime API.

The platform is implemented in Java, JSP, JavaScript and XML/XSLT. It is possible to extend the functions or integrate complex content by programming in JSP, JavaScript and XSLT - the developers of Time4You have proved this, with the efficient development of custom extensions.

However, to be able to make extensions, one needs to know (and have experiences) with the achitecture framework, which is considerably complex. The URL and JavaScript API is documented and examples are available. The XML-processing framework (XSLT-Pipeline) is not in the online documentation. The more advanced JSP API is dedicated to accredited developers, and not to end users.

- ein ganz wichtiger punkt bei der scorm/aicc-integration ist in unseren augen die möglichkeit, eigene content-produktionen scorm- oder aicc-fähig zu machen. dies erfolgt über den cmi authoring workspace im ibt skill management. Moreover, there is no technical documentation available for the JSP and JavaScript APIs, or for the technical framework (XSLT-Pipeline)?

extensible architecture
API available, but not publicly documented
support for client side scripting
Yes, see SCORM. Also, many functions are implemented in JavaScript and executed in the browser. Flash movies (also parametrized with XML files) work without restrictions.
SCORM Runtime API
Yes, fully supported
support for server side scripting
The system is programmed in JSP, which is therefore the most appropriate way to extend it
server side scripting API
Yes, but it is only partially available to end-users. Certified developers have a more privileged access to the 'internal' API
custom navigation scheme
-
Developers support
The web-interface of all tools is fully documented, and available as printed or electronic documentation.

Course designers, who want to go beyond static xhtml pages will have difficulties. A technical documentation of the JSP/JavaScript API is missing. The use of the XSLT pipeline isn't documented either, which makes it too difficult to modify the existing stylesheets, or add new ones. (A technical documentation should however be available to certified partners)

basic manual/tutorial for authors
Yes, the author's web-interface is fully documented
advanced manual for programmers
No, not publicly available
complete documentation of APIs
No, not publicly available
responsive support
-
online user forums
not available
Compatibility with common web authoring tools
Basically, any web content is supported in the content pages. Users have reported problems with intermixed use of the online xhtml editor and common web authoring tools - so one would have to decide for one editor.

The content pages are stored in the normal file system. Therefore it is possible to add a ftp- or WebDav-server to the machine, where the IBT-Server is running. This gives the authors transparent access to the content files.

compatible with standard web content
Any web content is supported. Users have reported problems with intermixed use of the online xhtml editor and common web authoring tools
up/download via ftp
yes, possible if a ftp server is installed
up/download via WebDAV
possible, if a WebDAV server is installed
Assessment
The quizz engine offers the most important quizz types, that can be automatically corrected: single choice, multiple choice, list maching, sort list, text answer. Various attributes (visibility, time limit, randomizations, ...) can be associated to single questions or question collections.

The web interface to create and edit questions and quizz banks is complex and not intuitive. New course designers are lost without documentation at hand and they initial training is essential.

Online Quizz Editor
Yes
Offline Quizz Editor
No.
Broad Range of Quizz Types
Basic types: single choice, multiple choice, list maching, sort list, text answer
Quizz Engine Extensible
Quizz Validity Feedback
-
Support for e-learning standards
The most important specification, SCORM is supported. There's also support for a subset of IMS metadata.

Entering a SCORM module does not necessarily open a new window, but can be embedded in the environment. This is an interesting and original implementation feature.

SCORM package import
Yes
IMS/SCORM-CP
Yes
SCORM-API
Yes
LOM Metadata
-
IMS-QTI
No
Adaptable look-and-feel
The look-and-feel can be fully customized using CSS stylesheets.

A more advanced approch is the adaptation of the system's XSLT stylesheets. The UI can be totally adapted (i.e. frame-free UIs are possible). Anything can be done - it only depends on the time/money that can be spent.

Adapting the UI requires a higly skilled XSLT and JavaScript expert/programmer, who also has intimate knowledge of the IBT-server architecture. It's also a problem, that the XSL processing framework is not documented.

course level look-and-feel
multiple corporate identities
Multilinguality
Multiple interface languages are supported: de, en, fr, it, es.
french environment
yes
german environment
yes
italian environment
yes
XML support
XML processing is a core technology of the IBT server. The system is extensively using XSLT to generate the user interface.

It is therefore possible and and even natural to represent content in native XML and providing XSLT stylesheets to display it. Unfortunately, there is no public documentation or tutorial available, that tells how new stylesheets can be integrated into the complex processing framework, and what the pitfalls can be. (Cerified partners do have access to a internal documentation)

support for XHTML content
Yes
includes XSLT engine
Yes
includes XSL-FO engine
Yes? (FOP engine of the Apache XML project)
Migration of current WebCT courses
-

System & Administration

System management
The installation of the system is impressively easy. Unpack the archive, edit a configuration file and run the server. Prerequisites are the freely available Tomcat servlet and JSP engine of the Jakarta project, and common database engine like mySQL, Oracle, MS-SQL etc.

The system requirements are moderate. Multiple instances of the server can run on a common notebook without problems. Medium sized installations up to a few thousands of users run on a single server with Tomcat and mySQL. Frequently accessed data like user account informations are fully cached in RAM.

Performance
Availability
Maintenance
Security
option: passwords are encrypted and transferred over ssl
Administration
The server is flexible enough to support multiple identities with different look-and-feel.

An internal API is available that allows to implement gateways etc. to make the authentication and user management system interoperable with other directory/authentication systems.

A weak point is the documentation: There's almost no technical documentation and just little admin documentation (that would i.e. be needed to make IBTServer interoperable with SWITCH-AAI)

One server - multiple institutions
Yes - multiple identities can be created
Flexible Authentication Scheme
Yes - the (internal) API allows to interoperate with external directory and authentication systems
Documentation
There's no technical documentation, and only litte administration documentation available. End-user documentation is available, however.
Flexible Licensing Model
The system is composed of modules (like plug-in components). These modules can individually be licensed for the number of expected users.

With regard to student numbers, different licensing models are available too.

Technical Architecture
The IBT server is entirely implemented in Java. Requirements:
  • Java Runtime Environment
  • Database: Hypersonic SQL (bundled with IBT-Server). Recommended database for production servers: Oracle, MS-SQL, MySQL (any DB with JDBC driver?)