Ghede
RPC for the unwashed masses
Crutcher Dunnavant
Trevor Jay
Sometimes called Baron Samedi, Ghede is the god of life and death in Hatian Voudou. Ghede stands forever at the crossroads between the Life and Death, and is always depicted as a skeletal black man, wearing a top hat and tails. He is the master of zombies, and Voudoun who wish to contact the dead must talk to Ghede as an intermediary.
Ghede is our solution to the general problem of system configuration and control. It provides an architecture for RPC mediation; designed to reduce the development costs of RPC clients and interface handlers to trivial levels. We believe that this will lead to rapid control coverage of any problem domain to which it is applied.
The problem of system configuration and control (SC&C), is the general problem of manipulating the state of a computer's resources. System configuration and system control are often discussed separately, and a great deal of effort has been applied to them individually, as well as together, but most of the software written towards creating general solutions to these problems has been trash; and even the well designed efforts have failed to succeed in any major way.
It is even difficult to make a strong distinction between system configuration and system control. System configuration is usually used to refer to manipulation of relatively static aspects of a system's state: the hostname, the IP, the services configured to run at boot up; whereas system control is usually used to refer to more ephemeral, and in general, immediate, manipulation of a system's state: restarting the webserver now, rebooting now.
One common approach to both problems is first to describe, or model, a generalized system. To then implement an object based interface to map between the model and the system; and to then provide methods for manipulating the idealized object systems provided. The effort involved in modeling, both in implementing the system mapping and in implementing clients which work with the model, is enormous. The effort involved in changing or extending an existing model is prohibitive.
“The three chief virtues of a programmer are: Laziness, Impatience
and Hubris.”
- Larry Wall
We think we can do better.
Crutcher worked for Red Hat as an OS Developer, involved in a modeling based SC&C architecture named 'Alchemist'. The code was completed as designed, worked, and released as LGPL'd code, which makes it free. It was extensible using pluggable modules, and merged configurations from multiple data sources. Despite a significant investment in implementation, not even Red Hat makes any significant use of the architecture whatsoever. Clearly, this was not due to availability (the library is free and open) or functionality (the library is stable and fully functional). We posit that the failure of the Alchemist is due to fact that the architecture is not compatible with the culture which it is intended to serve.
Dumb, stupid, poorly educated, and willfully ignorant people are the normal case. They do not model, they do not dot their “t”s or cross their “i”s, and they will not dig further into an interface than is necessary to implement their immediate needs. Taking the elitist position that these people should not be programing, is itself willfully ignorant of the state of the world, and of the cultural and economic realities which drive software development.
It is one of the axioms of the Ghede project that modeling, and the interface coercion which it inevitably requires, does not belong at the control level. We believe that different application domains have different ideal models, and the effort to create a model general enough to cover all application domains, even if it were successful, must yield a model too complex to be useful. Different models work better for different applications, and trying to reconcile them will always be very like religious wars. A good SC&C architecture should provide only an interface to configuration and control state, not a religious model interpretation. Given a raw interface to SC&C, any number of system object abstractions can be layered on top of it at the client level, exist in harmony, and not have to worry about the messy details. Further, for many application domains, raw interfaces are not only sufficient, but are the ideal solution.
We intend Ghede on being a general solution to the SC&C problem. As such, it must be available, people must be able to get it for development at no cost. It must be complete, it must provided the functionality necessary to perform SC&C; e.g. It must actually work. But it must also be used. System configuration and control does not exist in a vacuum, an ivory tower, or as a concept; any Correct architecture must achieve Coverage.
Coverage is a measure of the portion of the SC&C domain space which an architecture provides access to. The SC&C domain space for any modern system is so large that it is impossible for any single group or company, no matter how large, to achieve reasonable coverage alone. We feel that any successful SC&C architecture must be designed with coverage as its primary goal; and as such all design decisions must be made to encourage the implementors of independently developed components.
Given that we wish to maximize coverage, it is natural to examine the factors which limit coverage. We see that lack of availability and completeness limit coverage; no one will use an architecture they can't get, can't afford, or which just doesn't provide for their needs. Traditionally, complexity would also be considered a limiting factor, but we don't believe it is.
“For every problem, there is an explanation that
is simple, neat, and wrong.”
- H.L. Mencken, Prejudices
The world is complex, but most people get by fine by ignoring the vast majority of the details involved in their actions. They come in out of the rain, they drive on the right side of the road, they hold doors for strangers, and they don't think about why they do it. People follow rules, as long as the rules are easy to follow, no matter how complex the underlying reasons are, because they aren't thinking about them. We believe that complexity which can be ignored has no cost, and that complexity which must be understood has an extreme cost.
If we design to minimize complexity, then we will produce a proliferation of SC&C interfaces. But the tradeoffs necessary to achieve minimal complexity have always tended to force complex interfaces to solve the same problems in different ways; and this leads to more complexity which must be paid attention to.
If we design to minimize complexity which must be paid attention to, then we are designing for Negligence. We need not warp the system to have an artificially simple interface; if the paint by numbers way of using the interface always produces correct results, if ignorant programmers will always do the Right Thing the first time through, then we will have a proliferation of correct interfaces; even if the system is complex.
"If there are two or more ways to do something,
and one of those ways can result in a catastrophe,
then someone will do it."
- Murphy's Law
(correct version)
By corollary to Murphy's Law, if there is only one way to do something, it must be done correctly. Ghede's design uses the principles of bondage and discipline to remove choice from every arena where the existence of multiple choices is harmful, redundant, or simply unnecessary. We apply strict rules at every level of Ghede's architecture, as well as a suicidal fatalism. Incorrect behaviour must never be dealt with, it must be fatal. Ghede will not engage in error recovery or coercion, because we feel that it leads to situations where it is necessary. Any aspect of Ghede which can be specified is; even such things as version precedence, which Ghede makes no internal use of, so that there is no ambiguity in the development community.
“So it goes from God, to Jerry, to you, to the cleaners,
right Kent?”
- Bodie, Real Genius
The Ghede architecture leverages the XML-RPC protocol, and its large installed base of client libraries. It is designed to be simple to implement a Ghede client, if you already have the ability to implement an XML-RPC client. XML-RPC was chosen because it has been described as a protocol which is “so stupid it works”. The average programmer can read the XML-RPC specification, which is two pages long; and implement a fully conformant client/server pair; before that programmer could get all the way through a Corba tutorial. In fact, Ghede could be seen as a dialect of XML-RPC, it doesn't extend the syntax of the XML-RPC protocol, but it does specify the semantic meaning of method names and parameters.
Ghede is an XML-RPC server, and since XML-RPC is transported over HTTP, Ghede is a HTTP server. It accepts and validates XML-RPC calls on interfaces, authenticates and authorizes users to make those calls, and then runs an external handler to process the call. Residing on the local system, and run by the Ghede server, handlers need only read XML-RPC requests on their standard input, and write responses to their standard output. The call response is simply the inversion of this process, minus authentication and authorization.
Since handlers act essentially as filters they need only manipulate XMP-RPC documents and need not worry about the HTTP transport layer, authentication and authorization, or encryption. Additionally, since they are being run by the Ghede server, the server can drop permissions as appropriate on a per handler basis. As a result of this, handlers can be dumb.
There are many XML libraries, any of which provide an excellent base for developers to implement XML-RPC functionality. Clients and handlers could be implemented using any XML library with a reasonable amount of work. There are many XML-RPC libraries, any of which provide an excellent base for developers to implement the additional Ghede semantics. Clients and handlers could be implemented using any XML-RPC library with a small amount of work. We intend upon developing our own Ghede libraries to provide developers a base for implementing clients and handlers trivially. Additionally, we intend on providing BSD licensed example clients and handlers which can be cloned for incidental and correct implementation.
In keeping with our design for negligence, bondage and discipline; cloning will be the documented right way to begin a new handler. Due to the BSD license attached to these cloneable handlers they can be relicensed under any terms the developer wishes.
“The great thing about standards is, there are so
many to pick from.”
- Traditional
XML-RPC1 defines three types of messages: methodCalls, methodRespones, and methodRespone faults. methodCalls provide a syntax for specifying an interface the methodCall's methodName element; as well as a syntax for passing lists, structures, and arrays of lists, structures, arrays, strings, integers, floats, booleans, dates, and base64 encoded values. XML-RPC does not provide any semantics for the meaning of parameters or methodNames. methodResponses provide a syntax for specifying return values which is the same as the way in which calls pass their parameters; and methodResponse faults provide a syntax for returning an error integer and an error string.
A common extension to the XML-RPC methodName semantics is to declare that they are dot separated names in an object-like namespace, as in “foo.bar.baz”. Ghede uses this idea, but extends it with the idea of versions.
INTERFACE := NAMESPACE NAME ?('::' NAME) ':' VERSION
GROUP := NAMESPACE NAME ':' VERSION
NAMESPACE := ( | NAME '.') NAMESPACE
NAME := LOWER_ALPHA *(LOWER_ALPHA | DIGIT | '_')
VERSION := VERSION_TOKEN *( '.' VERSION_TOKEN )
VERSION_TOKEN := 1*(DIGIT | LOWERALPHA | '_' )
Ghede defines versioned interfaces upon an hierarchal namespace. An interface string such as “foo.bar.baz:qux0.3” defines version “qux0.3” of an interface “baz” in the namespace “foo.bar”. All versioning in Ghede is explicit, this permits unambiguous support for legacy clients side-by-side with newer clients.
Groups of interfaces tend to travel in packs; an example is our fictional “services.web.apache” namespace, with interfaces “start”, “stop”, “restart”, and “status”. It would be tedious to force client developers to program for one version of one interface and a different version of another; the programmer would be forced to check a reference manual for every call of every interface, to make sure that the correct version was being used. At any time in its development, we can expect there to be one collection of versions across the interfaces in the namespace which is considered the “current” collection, but it would also be tedious to force handler developers to iterate the version of all interfaces in a namespace, and ship both the old and new copies, just to keep the versions the same for client developers. Ghede's solution to this problem is to formalize the idea of a group versioned interface.
In Ghede, the group versioned interface “services.web.apache.start::apache:0.7” decodes to the version of the “start” interface contained in the “services.web.apache” namespace, and specified by version “0.7” of the “apache” group contained in the same namespace.
In no place are uppercase characters permitted in Ghede's names, namespaces, or versions; some languages which may need to bind to Ghede cannot support a distinction between upper and lowercase identifiers; and rather than have the names be case insensitive, we apply B&D here, and declare that they will be lowercase. Similarly, the files which specify the groups, interfaces, and namespaces are mapped onto a filesystem, and we wish to function properly upon case insensitive filesystems, such as OS X's.
Names must begin with a lowercase character, because some languages carry that restriction. No language which is in use today as a major implementation platform is thus unable to map to the restricted names which we use.
We have not yet detailed the extensions on the methodCall or methodRespone's parameter semantics, if there will be any. We foresee a period of tweaking and deep thought when we get to that level of implementation.
We have however detailed the semantics of fault methodRespones. Fault methodRespones require exactly two response values in the fault, an integer faultCode, and a string faultString. We will reserve all negative faultCodes for use as standard Ghede faults, such as 'InvalidParameters', 'AuthorizationFailed', and 'InterfaceNotFound'. faultStrings will be encoded in the form “<faultname>=<string>”, where faultnames will carry the same name restrictions which we apply to NAME. Handlers are free to create whatever custom faults they need, when Ghede's standard faults are insufficient. The standard Ghede faults are undeveloped at this time.
This fault mapping will enable the bindings for languages with exceptions to synthesize appropriate exceptions for Ghede faults.
If we choose to extend the parameter semantics, the most likely extension will be to require two base parameters, the first being a struct for Ghede meta-information, and the second being a parameter list of unspecified semantics, for the handler.
XML-RPC structs have named elements. This extension would specify meta-information fields which were permitted to be passed in this struct by the clients, and would enforce semantics as appropriate for all supported fields. Unrecognized fields would be dropped before the request was passed to the handler, and fields not specified by the client would be populated with their default values. Handlers would be able to depend upon any specified meta-information being present. This extension would NOT extend methodRespone parameter semantics, those would remain unspecified, and interface specific.
On the host side, our namespaces, groups, and interfaces map onto the filesystem in the following way: namespaces map to directories; groups and interfaces map to files in the directories as “<name>'.'<version>('.grp.xml' | '.int.xml')”. The group and interface files are in XML, the group files contain interface to version mappings, and the interface files contain instructions for running a handler for the interface. This ensures several important things:
Interfaces and groups must be unique at a particular version.
Packages of software will thus produce file collision if an attempt is made
to install conflicting versions of an interface
or group.
The collection of namespaces and their contents is human readable.
Since the files are in XML, external tools can easily reason about the contents
of the namespaces.
Additionally, each XML file must contain a 'doc' element, providing a documentation string for the interface or group. This is not optional, though we don't enforce that there be anything in the string.
Most users of the Python language wouldn't give up their __doc__ attributes, which are strings attached to all function objects. Combined with the interactive mode of the Python interpreter, this makes development fast. The programmer need not thumb through their API references, as a function __doc__ attribute can be directly examined in the interpreter environment with a simple “print func.__doc__” to the interpreter. Python was not the first language to introduce this, users of Lisp have had it for decades, which may explain part of the Lisp community's zealotry.
We will provide similar functionality in Ghede, by extending the previously detailed methodName mapping to add the following class of meta-calls:
'/'<type>'/'<meta-call>':'<identifier>
The types are 'n' for namespaces, 'g' for groups, and 'i' for interfaces. For example, the methodName “/i/doc:foo.bar.baz:0.3” maps to a documentation string request on the “0.3” version of the “baz” interface in the “foo.bar” namespace; and the methodName “/g/doc:foo.bar.quux:0.1” maps to a documentation string request on the “0.1” version of the “quux” group in the “foo.bar” namespace.
At this time, we foresee the need for only one additional meta-call, the “list” call on namespaces, which will list the namespaces, groups, and interfaces available in a namespace. But due to the nature of the meta-mapping, we can extend the calls at any later time, without fear of breaking existing call encodings.
While some RPC services provide open access to all resources, and rely on the network's firewalls to restrict access; this violates Ghede's Design for Negligence principle. Ghede must provide means of authorizing and authenticating users, so that handler's don't have to. Authentication is the process by which a user is positively identified; while authorization is the process by which it is decided if a user has the rights to use a particular resource.
We have investigated several authentication and authorization models. At this time, we have the following authentication mechanisms planned for implementation: HTTP MD5 passwords, and TLS/SSL certificates. Additionally, we would like to provide authentication by TLS + Kerberos, at some future date.
Our authorization model is based upon inclusion of permissions, and defines the following types of roles: User, System, and Versioned; which are stored as XML files. An authenticated user is mapped to exactly one user role, which details that user's permitted resources by listing an arbitrary number of system or versioned roles, as well as namespaces, versioned groups, and versioned or grouped interfaces. Versioned roles exist in the same namespace hierarchy as groups and interfaces, and their files are stored in the same filesystem mapping as group and interface files. Versioned roles may list any role, namespace, group, or interface in the role's namespace, or that namespace's children. System roles may list any anything in the root namespace, or any of its children; and they can also list other system roles.
Authorization is accomplished by walking the role map starting at the user's user role, seeking permission to execute the interface in question. Permission is granted if either:
An ancestral namespace of the interface is found in the role map, or
The versioned or grouped interface is found in the role map.
Of particular interest is that if an call is being made on a versioned interface, and a group is listed in the map which contains that versioned interface, this is NOT sufficient to grant permission. This is not simply a matter of efficiency, but of policy. If programmers are instructed to program for a given group, they cannot leave that group, even if they know the versions.
Significant improvements over the naïve map walking algorithm are available, the simplest of which is to not descend the map down any path which is not ancestral to the interface being checked.
The intent of this authorization mapping scheme is to simplify the development of handlers, packages of handlers, and system permissions. Domain groups should have roles, for example a “services.web.apache.apache:0.3” role. System roles should collect permissions for domain specific interfaces, which they can do by including group roles for the interfaces in the domain, without listing all the interfaces in each group. For example, the “web_admin” system role would probably include the apache role given above. This permits User Roles to be very simple, usually including only a few System Roles.
Ideally, only User Roles should be altered at any local site. If a need is felt for groupings not collected in any System role available on the system, a site local System Role can be defined, but the existing System Roles should usually not be altered. Versioned Roles should NEVER be changed at any local site.
If shipped System and Versioned Roles are not changed, then they will upgrade cleanly with packaging systems, allowing a User Role which includes the “web_admin” system role to continue to work “Right” when the “web_admin” role includes a new version of the apache group role at upgrade time.
This bridge will only take you halfway there
To those mysterious lands you long to see:
Through gypsy camps and swirling Arab fairs
And moonlit woods where unicorns run free.
So come and walk awhile with me and share
The twisting trails and wondrous worlds I've known.
But this bridge will only take you halfway there-
The last few steps you'll have to take alone.
- The Bridge, Shel Silverstein
The Ghede server is being implemented with the following goals in mind:
Hackable – the server code base must be easy to read, modify, and must be organized to support multiple developers working simultaneously with minimal version repository collision.
Portable – the server code base should be trivially portable to any POSIX.1 system.
Small and Fast – the server should perform as quickly as possible, while utilizing minimal system resources. This program is meant to be a control daemon, and as such will always be running on host systems. Additionally, if we can keep it efficient, we can port it to embedded and portable devices.
Works Out of the Box – the code base should ease the process
of compilation and packaging on any platform, and the server should require
no configuration changes by the packager or user to do the Right Thing on
any supported platform.
(It should be as far away from innd as possible.)
The server is implemented in C, and will make use of open, free, and portable XML, encryption, and option parsing libraries, but otherwise it will do everything itself, for speed, portability, and memory footprint.
We have applied B&D to our own development, and have taken a few divergences from normal source layout. Modern computers have plenty of space-time to work with for a compile cycle, and we have taken advantage of this to simplify header file inclusion.
Variable aspects of the code's build process, such as the default directory for the xml namespace tree, which compiler to use, if debugging is to be compiled in, if the server binary is to be stripped, and the maximum size of various cache lists, are specified in the file “ghede_config.h”. This file includes a file “config.h”, which is where compile time changes are to be put to these variable portions of the build system; these changes override the defaults set in ghede_config.h. There is a file, “Makefile.vars.source”, which itself includes ghede_config.h, and is processed with cpp to produce the file “Makefile.vars”; which is included by the Makefile. In this way, build options changed in config.h are available to make, and all aspects of the build depend only upon config.h.
All headers for the project are included in one master header, “ghede_common.h”. All .c files include first ghede_config.h, then the library headers which they need, and then ghede_common.h. While this means that every single .c file must include all project headers, but in practice on a modern system, this produces only about a quarter second additional delay, so it's completely acceptable.
The largest divergence we've taken in our code development is that the .c files do not themselves contain function definitions, but include one file for each function. If there is a .c file of the form “ghede_foo.c”, there will also be a directory of the form “ghede_foo.c.p” (.p for portfolio), which, for every function which would normally be defined in ghede_foo.c, contains one .c file of the form “ghede_foo_<function>.c”, which is included by ghede_foo.c. This permits version tracking on the changes to each individual function, and dramatically reduces version repository collisions when multiple developers are working on the source base simultaneously.
This source organization, while admittedly a bit odd; has worked extremely well in practice. Supported by automatically generated build dependencies, and file templating tools, development is speedy. After an initial period of adoption, developers have tended to pick up everything pretty fast, and start wondering why you wouldn't do it this way. They tend to ask the question “When did this stop being crazy?”.
We've fully instrumented the code. This means that every single function entrance and exit is wrapped by tracking macros. These macros disappear if debugging is turned off, but when debugging is on, it is trivial to track the call path of the program, as trace information is written to standard error if GHEDE_DEBUG_TRACE_LEVEL is set higher than 0 in the environment. The trace entrance functions appear in the form:
ghede_trace_enter \
("$Date: 2002/11/21 23:59:38 $"
"$Revision: 1.4 $"
"$Author: crutcher $");
These are magic CVS tags. They are modified by CVS upon commit, so that the function will always contain the last date of its modification, the revision number of the function, and the person who performed the change. Tracing can be told to display this information in its trace output, so it is very easy to assign blame for broken functions.
The trace layer uses a logging layer; which provides 'fatal', 'error', 'warn', 'info', and 'debug' levels, in order of precedence. The log layer can be set at a log level, and only messages at that level or higher will be logged.
For command line parsing, we use the “popt” library by Eric Troan. It is all-singing and all-dancing, and really simplifies argument parsing. This will limit our portability to those platforms where popt compiles, but since that's about everywhere, we don't view this as a serious limit on portability.
The core of the Ghede server is the IO multiplexer, a structure which contains a list of iom_jobs, which themselves contain pointers to function tables and a pointers to job data. Every time the multiplexer iterates, it walks its job list, and for each job which is “awake” (determined by a state variable in the job structure), it calls a “register” function on the job, as specified by the job's function table. Every class of job provides different functions for its table, and a job's register function is responsible for telling the multiplexer which file descriptors it wishes to watch for read, write, and exception availability, as well as what wait timeout it wants, if any; the multiplexer keeps the shortest timeout specified.
The multiplexer then calls pselect upon its file descriptors, with its timeout. It then calls time, and stores the result in the multiplexer structure, since many of the jobs can be expected to want to know how much time has passed, and we can cache that and prevent system calls in the clients.
The multiplexer then calls the “process” function in each job's function table on all jobs which are awake. Job's process function can determine if the file descriptors they care about became active by a check call on the multiplexer, and can ask the multiplexer what time it is. Jobs can also put jobs to sleep, wake them, and kill them; either themselves or other jobs.
The multiplexer then iterates until it has no more dead jobs, calling the delete function in each dead job's function table. The iteration at this point is due to the fact that a job might kill another job on delete.
All of this works, because all jobs use non-blocking IO on all file descriptors. This permits a single process to do a rather large amount of work, with out the overhead of inter-process communication, or inter-thread synchronization and locking. The non-blocking setup calls, and the pselect call, are the most likely source of changes at port time.
The HTTP layer is partially complete at this time. We have a correct RFC20682 header parser, a correct RFC2068 HTTP scheme URL parser, a layer which performs raw socket operations, and a layer which performs the HTTP protocol operations. The reason for the separation of the layers is that we will need to add the TLS socket layer at a later time, and with this separation, only minimal changes to the HTTP protocol layer will need to be made to enable it to support both raw and TLS sockets.
At this time, the HTTP layer does not support connection timeouts, and the raw layer does not support socket exceptions. The combined layers at this time will return a 501 Method Not Implemented error on all requests.
The next stage of work at this layer is to produce a GET method handler, and then a POST method handler, which will reuse most of the code in the GET handler. XML-RPC is transmitted over HTTP POST requests.
The TLS (Transport Layer Security) layer is not yet written. It will provide encryption for the server. We intend on using the GNUTLS library for this part. It is open, and has received good words in the community. We intend on staying away from OpenSSL, mainly due to the fact that the official position of most OpenSLL hackers is “it sucks”.
Ghede is a server, and Ghede is a parser, and both of these mean buffers. We've got a very robust chunk buffer layer, which provides all kinds of nice features beyond what is usually found in a chunk buffer. Our chunk buffers track the portion of the first chunk which is available for reading, and the portion of the last chunk which is available for writing, this permits read and write calls with no copies, as shown in this example:
while (cbuf->len) {
ret = write(fd, cbuf->out_buf, cbuf->out_buf_len);
if (ret >= 0) {
ghede_cbuf_consume_out(cbuf, ret);
} else switch (errno) {
case EAGAIN:
case EINTR:
continue;
default:
/* an error occurred on the fd
*/
goto error;
break;
}
}
if (0) {
error: do_something();
}
This has greatly speeded the development of all layers which make use of the buffers.
Our authorization mapping requires a great deal of name and version string comparisons, and our message layers perform a good deal of string passing. In order to simplify memory management, and to speed string comparison, we've implemented a refcounted string internment system.
Our string interment layer is based upon arenas and refcounts. A string may be interned in an arena, and the internment token returned will always be identical to all other strings interned in the same arena. We have arenas, because we string an additional pointer onto each internment token, which allows custom purpose arenas. 'ref' and 'unref' operations can be performed upon a string, and the token will be dropped from the arena when the reference count reaches zero.
Name resolution uses a custom purpose arena, and the name tokens carry pointers to their parent namespaces. Thus, the token for the name “foo.bar.baz” carries with it a pointer to the token for “foo.bar”. This reduces ancestral checks for namespaces to a very, very cheap pointer walk.
Future work on the internment layer is planned to extend the semantics of token request. If a token is requested on a NULL arena, the internment layer will construct a token which is not attached to an arena. No speed improvements are available for checks on string equality in this case, but refcounting is still done. This will be the uniform storage method for non-io-buffered strings throughout the code base.
Our XML-RPC layer is incomplete, though it is significantly far into its implementation. We are using the SAX parser interface of the libXML library from the Gnome project, which is very fast and very portable.
Our Authorization mapping layer provides a sticky problem. The filesystem semantics of our authorization map are easy for developers to work with, but they prove complex to resolve authorizations upon. We are currently caching authorization map file lookups internally, and parsing those files using the SAX interface of libXML. This layer of the authorization map is complete, though the authorization resolver, which walks the mapping, is not.
Long term, we will probably have to move to a precompiled authorization cache database, built from the XML authorization maps on the filesystem, for speed; but we foresee this as being a project for the far future. Our cache layer should provide for most of the speed we need at this time in development, and even in initial deployment.
The handler call layer is not yet written. It will take an interface map, create a pair of communications pipes in the multiplexer, and spawn a child to run the interface. The child will close all of its file descriptors, except for its pipe descriptors, which it will dup2 to stdin and stdout. It will then manipulate its environment and permissions, as described in the interface specification file, and then exec the handler. The actual contents of the interface map will contain a variable section, which will have different semantics on different operating systems, as permissions are not uniform across operating systems.
Obviously, the primary focus of current work is in completing the server. We estimate we'll need about six months to get it to the point where it is done, but that time will overlap some with initial handler and client development; as some aspects of the call interface need to be tested in several different ways, to see what works best, and what simplifies handler and client development the most.
We intend upon providing C, C++, Python, Perl, and Java client and handler libraries, which make use of the best-of-breed free XML-RPC layer libraries we can find for each language, and extend them with the Ghede semantics. These libraries will be licensed with the LGPL license, which ensures that the libraries stay free, but that programs which use them may be under any license.
Developed in parallel with each handler library, we intend upon producing an echo handler, which only echoes its parameters, in each language. These echo handlers will be licensed with a BSD style license, so that developers are free to clone them and change the license. The encouraged style of development of a new handler will be: pick a language, clone the echo handler for that language and its interface specification, alter the handler and specification to suit your needs.
We then intend upon writing a collection of handlers for the general tasks encountered on a modern *NIX system, such as starting, stopping, querying, and listing running services, changing run levels, and the like. By the time we reach this level, we will be seeking adoption in the community of *NIX developers, starting with the Debian project.
We also intend upon writing a “shell binding handler”. This handler would provided different behaviour for different interface specification files. The specification files would provide the handler with an encoded shell command on its command line. The encoding will specify parameter semantics which the handler is to accept in its Ghede calls, and substitute into the shell command, and how to return the output of the command. A completely fictitious command line example follows:
-c “rpm -Uvh \”%1\”” --params s --fault 1 RPMError
Run the command “rpm -Uvh \”%1\”, substituting the first parameter in for %1; expect one parameter, a string; and raise a fault of type RPMError if the exit code is 1.
Such a binding handler would provide many interfaces for one program. While this wouldn't give the most robust parameter semantics, it would make it easy to provide a new interface quickly for quick-hack purposes.
We have deliberately avoided the complexities of CORBA, Sun RPC, ASN.1, SOAP, and the like in the development of Ghede. But once Ghede is complete, it would be ideal to provide gateway layers which would allow CORBA, RPC, and SOAP clients to talk to Ghede.
Our first non-Linux port is planned for OS X, and then the BSDs. They are all as close to Linux as it is really possible to get and not be Linux, and we foresee no problems with this port. We have not done anything in the server or filesystem mappings which we think would prevent a Windows port, and that is planed for the second porting effort.
“It always starts small ...”
- Egg Shen, Big Trouble in Little China
The most exciting Ghede application we've conceived of at this time is the development of user control interfaces written in XUL for Mozilla. Mozilla is not really a web-browser, it is actually a user interface interpreter, and its interface language is XUL, the XML User interface Language. Additionally, Mozilla already has an internal XML-RPC layer. Since XUL interfaces can be downloaded like all other web content, we for see extending the Ghede server to act as a basic web file server in addition to acting as an XML-RPC server. This will allow us to server a user interface to a client machine, which loops back through Mozilla to perform Ghede calls on the serving host. Any computer with the Mozilla web-browser will be able to control and configure the services provided by the interface.
We intend upon this being the main focus of our future work with the development of user interfaces which utilize Ghede, and we are both very excited by the possibilities this creates.
1http://www.xmlrpc.com/spec
2http://www.ietf.org/rfc/rfc2068.txt