Sound and vision

The web used to be a static place. Web servers used to just sit theree birth of streaming technology and the arrival of new server APIs has taken the art of websites into another dimension. Simon Bisson looks behind the technology. at the end of their leased lines, delivering HTML documents. But now things have changed. Websites have become more dynamic and are gearing up to be the medium for a new generation of broadcast technology.

There have been three generations of websites. The first used static HTML to create web presences, the second added multimedia and simple forms to capture user information and allow enquiries. The third generation has gone a stage further, turning the web page into a dynamic interactive environment. Sites have stopped being just web pages; they're now web applications. Running a dynamic website may solve many of the problems of web publishing, but it can also add some fresh ones.

Dynamic web applications aren't the only way to add interactivity to a web page. Instead of delivering images as GIFs and JPEGs or using ShockWave to download animations, it's now possible to deliver a realtime stream of sound and pictures over the internet. Using streaming multimedia, the web server can become your own TV or radio station.

Personalised web visiting hours

Increasingly websites are moving away from the traditional web server model, and towards a model more akin to dynamic data-driven web applications. Many businesses have been influenced by the big online services like AOL or CompuServe; instead of seeing a corporate home page, a visitor receives a personal greeting and a list of pages that contain information they may be interested in, as well as a list of new content. Where they might have once delivered individually created web pages, sites now generate HTML on-the-fly. Information stored in databases or server applications is used to create pages that are relevant at a specific point in time or to a single user.

The technology used to create dynamic web content has been around almost as long as the web server itself, and is a direct descendent of the familiar CGI programming techniques used on thousands of websites. In fact, the first dynamic sites used CGI to deliver page content based on HTML form submissions. Most web servers have CGI support, and when combined with a powerful scripting language like PERL, connection to a wide range of external databases is as easy as downloading the required libraries from the CPAN PERL archives. Using PERL to deliver dynamic content can be a problem on busy sites, as each CGI script called will launch a separate copy of the PERL interpreter. If you're using the popular freeware Apache web server, you can install PERL as a server module, so that the interpreter is always in memory - speeding up your CGI scripts.

Second generation web servers added tools that allowed web developers to build direct links between web servers and back-end systems. The new server APIs could be used with tools like Visual Basic to build web applications and dynamic sites. Instead of working through a slow CGI interface, web servers could now use libraries and executables that ran in the same memory space as the server - so one copy could be used by all the users of a website. Together with the release of the 2.0 browsers, which introduced cookies that allowed a user's preferences and other server-specific information to be stored in the browser, websites could become more interactive - as well as making it easier to store information on users' visits.

Tools like HAHT's HAHTsite and Allaire's Cold Fusion take advantage of these APIs, and give you a quick way of adding database connections to your sites. You will need to train developers to use these tools - but as HAHTsite uses a dialect of Visual Basic and Cold Fusion's CFM mark-up language is very close to HTML, it's easy to transfer skills. You'll also need to add memory to your server for these to work effectively.

Database connections can be local or external, and you can use either native interfaces or ODBC connections to link into databases. If you're using Apache, then the PHP module gives you the tools to add scripts that link to the free MySQL database.

Both Netscape and Microsoft have also added web application suites to their servers. With these you can turn your web pages into programs, with scripts embedded in your HTML. Netscape uses JavaScript in its LiveWire application, with Microsoft's Active Server Pages (ASP) using an open scripting interface. ASP comes with JScript and VBScript as default scripting languages, but versions of other popular scripting languages are available - including PERL and Rexx. Both environments provide tools for connecting to databases, but debugging can be a problem. There are issues with using both these environments on busy servers, as well as developing complex database queries. You may find it easier to use stored database procedures rather than direct SQL queries, especially when working across multiple tables.

Microsoft advises against using ASP for direct database applications for more than 200 concurrent sessions, as there are problems with threading database connections and hosting session objects. You can improve performance by using NT's Transaction Server (included in the NT4 Option Pack) to host database connection objects by including them in COM objects written in Visual Basic or Visual C++. Making a web application part of a distributed application environment helps improve scalability - to improve performance, just add more servers to host the Transaction Server components. The combination of ASP and the Transaction Server is the basis for Microsoft's DNA (Distributed interNet Applications) architecture.

You can use Notepad or any other text editor to build LiveWire or ASP applications, but you'll find that it's easier to use specialist development tools. Microsoft's Visual InterDev is one of the best around, and it's part of the Visual Studio development suite. Using Visual Studio you can build multi-threaded COM components that can be used to create Transaction Server packages. You'll need to upgrade Visual Studio with Service Pack 3 if you're going to use Visual Basic to create COM objects, as otherwise Visual Basic doesn't support a shared memory threading environment.

Netscape's equivalent tool, Visual JavaScript is written in Java, and can be rather slow. However, it gives you the tools to build database connections using LiveWire's built-in objects, as well as working with server-side JavaBeans to create your own components.

Jump-starting the mainframe

The mainframe isn't dead - it's just been renamed the Enterprise server.

Mainframe data and applications are often the heart of a dynamic website, especially for those delivering financial data. You can access some systems through an ODBC database connection, but if you're connecting to a CICS application or a green screen system this isn't possible. There are tools that turn a mainframe running CICS into a web server, but these often don't allow you to change the look and feel of your pages. An alternative solution comes from host connectivity specialist Attachmate. Using Attachmate's Host Publishing system you can embed connections to mainframe green screen applications in ASP web pages running on an NT server. Microsoft now offers an SNA connection tool that will enable you to build CICS transactions and embed them in Transaction Server packages.

An alternative approach to creating a dynamic website is to use a web application server. These cover everything from Java application servers to tools that add web frontends to enterprise distributed applications that have been built with object-oriented systems like NatStar and Forte.

By using a distributed application environment, a dynamic web application can take advantage of its scalability and reliability. The latest generation of mid-range web application development tools, like NetDynamics Application Server, include Integrated Development Environments and database connection wizards, as well as Corba support for links to external applications.

As these tools offer scalability, you can add extra hardware to improve performance. You'll also need a more powerful server, especially if you're going to run the web server and an application server on the same hardware.

If you're running the website for a newspaper or a magazine, then you're going to want to control a large amount of content - content which can change from day-to-day or hour-to-hour. The technologies used to deliver dynamic websites can also be used to control and manage web content. Systems like Vignette's StoryServer use dynamic database-driven publishing tools to control content, and to deliver it to the browser. As more systems and packages start to implement XML, it will become easier to write your own metadata-driven content control systems.

Going with the flow

One thing guaranteed to catch someone's eye is movement. Animated GIFs and ShockWave can give a site the appearance of dynamic content, but the short animation loops soon give themselves away. Streamed multimedia can make a big difference - instead of a GIF of your company's MD, you can show a video of his latest speech, or use audio to deliver a welcome message.

The first big player in the streamed multimedia market was Progressive Systems. The free RealPlayer software, distributed with web browsers, allowed it to quickly develop a presence in the web media market. With Microsoft and other companies entering the streaming marketplace, Progressive became Real Networks, and started to focus more strongly on content. However, it still produces its popular RealServer streamed multimedia server, which is used by TV and radio stations all over the world - including the BBC.

Implementing a RealServer solution can be expensive - a 100-stream server will cost you nearly $6,000 (#3,750), with support and upgrades for a year at an extra cost of $2,400. A new version of Real's software is currently under development, and has just entered public beta. RealSystem G2 adds text and picture streams to the audio and video, with the RealPresenter toolkit helping you build presentations for internet delivery. You can use this technology to create interactive sales pages, as well as distance learning applications.

Microsoft's alternative is NetShow and the software giant is now including the NetShow client in the latest release of its multimedia player. Like the RealServer, the NetShow server is a tool for delivering streams of audio and video data, using Microsoft's ASF format. A more powerful version is the NetShow Theatre, which allows you to build a video-on-demand system able to deliver MPEG video from a cluster of NT servers. Microsoft provides an online tool on the NetShow website to help you decide on the size of your NetShow Theatre server. You'll find that to get the best performance you'll need to use at least a 100Mbps network connection.

If you're setting up a server to deliver streamed multimedia, server hardware will not be a problem. The current RealServer will run on a 16MB Pentium taking up only 2MB of hard disk space. You'll only need 3MB of free memory to run the server - plus up to 20KB for each audio stream and 60KB for video. Little disk space is required to hold the files, at the highest quality (80Kbps) one hour-long audio stream will only take up 29MB. However, you will need plenty of bandwidth between your server and the nearest IP peering point to give the best service to your users, as very few streaming protocols support the more bandwidth-efficient IP multicast.

You don't need a streaming media server to deliver streamed content over the internet. The latest release of Apple's QuickTime multimedia file format, v3.0, allows you to create files that can be placed on a standard web server for delivery over an HTTP connection. The file format is designed so that the player in a user's browser can adjust its performance to simulate a streamed content delivery, reducing image quality to cope with slower links. Similar tools are included in the latest release of Macromedia's ShockWave for Director.

The World Wide Web Consortium is also currently working on the definition of a Synchronised Multimedia Integration Language (SMIL). This is designed to go beyond traditional HTML by adding in-time dependency. This is done with synchronisation tags that allow the definition of parallel and sequential events. With a time-based event model it will be possible to use SMIL to deliver a continuous multimedia presentation, scripted using SMIL tags.

By linking SMIL to a commonly agreed streaming protocol, the cost of implementing a streamed multimedia system is likely to drop, as all the software you'll need to deliver streamed video and audio will be built in to your web server.

This drop in price is likely to ensure that dynamic content on the web continues to grow in popularity. The ultimate question though is, 'how much bandwidth-hungry multimedia streams can you feed to users who are stuck on low bandwidth connections?'