playListNetWork Authoring Environment
2. technical innovations.
This code incorporates some interesting technical innovations that
have not appeared in any comparable program.
2.1. Promiscuously add properties to any clip.
Any artist can add any type and name of property to any clip, even
clips they don't own. In other words, you can annotate remote
objects, even on someone else's server.
2.2. An extensible XML language for the representation of edits on
We have an XML language named eel that represents a set of
operations, edits, on media assets and playlists. Instances of the
program communicate between each other by sending messages in this
This gives a unique representation for every action of an editing
session on media assets.
This also gives us the capability to create a language to remotely
manipulate GUI objects. We in fact do this in the current program by
using the language to remotely control the Java Abstract Windowing
Toolkit (AWT) objects that make up our interface.
2.3. A complete
record of every operation.
We store every operation ever performed on the system which makes it
possible to revert the system to its state at any given point in
time -- "infinite undo" -- though at this time it has to be done
2.4. A sophisticated,
extensible query language.
Within eel, there is an XML language for representing queries on the
database. This not only lets you perform all the searches that SQL
would allow but the following additions:
-- keywords; regular expression ("wildcards" and more)
-- wildcard matching on any type.
-- arbitrary logical expressions (multiple AND, OR, implies)
-- special typed operations on Strings, Dates, Times, TimeIntervals
In addition, new search query tags can easily be added to the system
to add new search types -- to search external objects, even to do
image recognition processing. See the next paragraph.
2.5. A fast, customizable
serialization ("marshalling") system that
allows for disparate objects to be sent over the same wire or stored
in the same database.
Eel relies on a custom XML serialization mechanism relying on a
fairly simple abstraction of objects into having named data and a
list of serializable children.
However, the serializable class does not have to be in this format --
it merely needs to expose a proxy that does.
This allows us to deal with a lot of problems that other
serialization mechanisms don't handle, such as immutable objects,
singletons, or purely "in-program" objects (objects that can only
exist while the program is running).
This also allows us to add new tags representing new classes of
objects or new searches into the system very quickly.
2.6. A completely
queue-based distributed editing system.
PLNW is a highly distributed system. It relies on a completely
queue-based synchronization to avoid any possibility of deadlock or
other forms of loss of liveness.
Each operation is sent as an indexed piece of Eel, a fragment of XML
with a number of it. At each point where there is communication,
there is a synchronized queue that forces a definite order onto the
The server's input queue is the final arbitrator as to the order of
operations. It assigns the order of operations. Clients can attempt
to execute operations but are unable to assign a definitive index
number to an operation till the server has seen it and indexed. This
might result in a client having to reorder its operations.
(There is in fact no necessity for the server to be distinguished in
this fashion -- later versions could use a distributed editing
conflict resolution mechanism which would allow the system to
function even in the absence of a server or to form ad-hoc
playListNetWork networks -- the decision to use queues means that we
can switch our conflict resolution strategy without changing any
other aspect of the code.)
3. How it works.
3.1. The application.
This program is a Java application. While we are demonstrating this
system on a Windows platform, we have made earlier builds for the
Macintosh and there seems to be no reason why we could make this run
on any of the many platforms that Java supports since we have only
used crossplatform features of the language.
3.2. The GUI.
The GUI is Java's Abstract Windowing Toolkit which is fully
cross-platform. We deliberately avoid using the advanced features of
Java's Swing system for many reasons, including speed and simplicity,
though it would be perfectly reasonable to consider using Swing or
SWT or some other advanced GUI system for an earlier revision.
3.3. Network communications.
The network communications layer is critical. An early attempt made
use of Sun's JXTA system but this would have introduced great layers
of complexity into the system and was slow and huge.
We use a system where raw sockets in a sandbox are wrapped with a
single thread each. The XML parser or generator speaks almost
directly to the input or output socket -- there is only a single
memory buffer in each direction which is necessary for network
efficiency. These threads run at high priority and merely process
the incoming XML and throw it onto a queue, where a lower-priority
thread will actually execute it.
A keep-alive thread on the client-side only detects if the socket
dies and promptly opens another one. While we have not been able to
test it in most of the possible cases, we could code this to handle
almost any possible exceptional condition on the network.
The result of this is that each socket/thread combination occupies
more memory, though significantly less than 100K per user connection
in the server in the worst test case. That means we have less than
1M of memory overhead per 10 connections in the server, not a large
price to pay on modern machines with thousands of megabytes of memory.
3.4. The object
None of the media objects reside in memory -- they are all
represented as external URLs on some server. That means that the
representation of a media object is only a few dozen descriptive
properties that can be encoded in text, rarely as large as 1K.
As a result, we are using an in-memory database where all the objects
live in memory at one time. Periodically or at shutdown time the
program stores all these objects to a flat file on the computer in
We store a journal of each operation to disk as we go, also as a
series of Eel commands, so we can recover from server failures by
loading the most recent checkpoint and applying the most recent
journal to it. Writing to the journal is extremely fast, much faster
than a random-access write to a database file, because we are only
appending to the end of a file that we keep open for write-only at
And there are many advantages to using an in-memory database that
make this checkpoint/journaling schema worthwhile. The speed of
searches is extremely fast of course, but this also means that we can
perform almost any search we can conceive of, since all the objects
are "right there" -- we aren't at all constrained by SQL. We don't
have to worry about the issues involved in storing live objects to a
database (like circularity, concurrency issues and such) as we only
store and recall when the program is starting up.
We make use of this in two ways in the program:
we allow powerful "regular expression" searches
http://etext.lib.virginia.edu/helpsheets/regex.html is a primer
is a list of all the patterns we support, with the exception of
the "Groups and capturing section", where we will match the patterns
correct but of course have no way to return the captured results.
and we allow arbitrary search plug-ins, demonstrated by the date
and time searching and advanced logic searches.
Sadly, we had to hack an open-source XML parser slightly to get what
we needed as standard XML parsers only expect a single XML fragment,
not a stream of them.
We use the extremely small and fast open-source NanoXML parser and
jiggered it to accept a stream of XML fragments which it spits out to
NanoXML is a standard SAX 2 parser.
While DOM parsers are very popular, SAX parsers have the edge for
this sort of problem -- the DOM parser must read the entire XML
fragment into memory at one time and then send it to the client,
whereas the SAX parsers have no real knowledge of the document's
structure and instead call back to the originating program for each
language element, perfect for what we are doing. In fact, we don't
actually create language structures for the XML objects, instead
reading or writing them right from or to the buffered socket), which
means that we can pass large objects through the system without ever
making more than one copy.
The NanoXML parser is NOT a validating parser, which means it does
not check the validity of the input against a fixed grammar. This is
fine for us as we have not had the leisure to create a grammar -- the
XML language is 'whatever the program uses'. This is fine as it's
not intended to be read by humans (though it reads quite well) --
it's only intended to be sent back and forth between various
instances of the program.
Eventually, the language will become mostly fixed and then it becomes
worth our while to create a grammar and switch fairly easily to a
A neat and useful configuration system allows the program to read
startup configuration values from a human-readable and editable text
program -- each value has a default and you can override these
defaults by making a new entry in the configuration file. This is
how we "install" the program.
[Tom Ritchford, May 2004]