CMUCL offers a facility for interprocess communication (IPC) on top of using Unix system calls and the complications of that level of IPC. There is a simple remote-procedure-call (RPC) package build on top of TCP/IP sockets.
The
remotepackage provides simple RPC facility including interfaces for creating servers, connecting to already existing servers, and calling functions in other Lisp processes. The routines for establishing a connection between two processes,
create-request-serverand
connect-to-remote-server, return
wirestructures. A wire maintains the current state of a connection, and all the RPC forms require a wire to indicate where to send requests.
Before a client can connect to a server, it must know the network address on which the server accepts connections. Network addresses consist of a host address or name, and a port number. Host addresses are either a string of the form
VANCOUVER.SLISP.CS.CMU.EDUor a 32 bit unsigned integer. Port numbers are 16 bit unsigned integers. Note:
portin this context has nothing to do with Mach ports and message passing.
When a process wants to receive connection requests (that is, become a server), it first picks an integer to use as the port. Only one server (Lisp or otherwise) can use a given port number on a given machine at any particular time. This can be an iterative process to find a free port: picking an integer and calling
create-request-server. This function signals an error if the chosen port is unusable. You will probably want to write a loop using
handler-case, catching conditions of type error, since this function does not signal more specific conditions.
create-request-serversets up the current Lisp to accept connections on the given port. If port is unavailable for any reason, this signals an error. When a client connects to this port, the acceptance mechanism makes a wire structure and invokes the
on-connectfunction. Invoking this function has a couple of purposes, and
on-connectmay be
nilin which case the system foregoes invoking any function at connect time.
The
on-connectfunction is both a hook that allows you access to the wire created by the acceptance mechanism, and it confirms the connection. This function takes two arguments, the wire and the host address of the connecting process. See the section on host addresses below. When
on-connectis
nil, the request server allows all connections. When it is non-
nil, the function returns two values, whether to accept the connection and a function the system should call when the connection terminates. Either value may be
nil, but when the first value is
nil, the acceptance mechanism destroys the wire.
create-request-serverreturns an object that
destroy-request-serveruses to terminate a connection.
destroy-request-servertakes the result of
create-request-serverand terminates that server. Any existing connections remain intact, but all additional connection attempts will fail.
connect-to-remote-serverattempts to connect to a remote server at the given
porton
hostand returns a wire structure if it is successful. If
on-deathis non-
nil, it is a function the system invokes when this connection terminates.
After the server and client have connected, they each have a wire allowing function evaluation in the other process. This RPC mechanism has three flavors: for side-effect only, for a single value, and for multiple values.
Only a limited number of data types can be sent across wires as arguments for remote function calls and as return values: integers inclusively less than 32 bits in length, symbols, lists, and
remote-objects(see section 9.1.3). The system sends symbols as two strings, the package name and the symbol name, and if the package doesn't exist remotely, the remote process signals an error. The system ignores other slots of symbols. Lists may be any tree of the above valid data types. To send other data types you must represent them in terms of these supported types. For example, you could use
prin1-to-stringlocally, send the string, and use
read-from-stringremotely.
The
remotemacro arranges for the process at the other end of
wireto invoke each of the functions in the
call-specs. To make sure the system sends the remote evaluation requests over the wire, you must call
wire-force-output.
Each of
call-specslooks like a function call textually, but it has some odd constraints and semantics. The function position of the form must be the symbolic name of a function.
remoteevaluates each of the argument subforms for each of the
call-specslocally in the current context, sending these values as the arguments for the functions.
Consider the following example:
(defun write-remote-string (str) (declare (simple-string str)) (wire:remote wire (write-string str)))The value of
strin the local process is passed over the wire with a request to invoke
write-stringon the value. The system does not expect to remotely evaluate
strfor a value in the remote process.
wire-force-outputflushes all internal buffers associated with
wire, sending the remote requests. This is necessary after a call to
remote.
The
remote-valuemacro is similar to the
remotemacro.
remote-valueonly takes one
call-spec, and it returns the value returned by the function call in the remote process. The value must be a valid type the system can send over a wire, and there is no need to call
wire-force-outputin conjunction with this interface.
If client unwinds past the call to
remote-value, the server continues running, but the system ignores the value the server sends back.
If the server unwinds past the remotely requested call, instead of returning normally,
remote-valuereturns two values,
niland
t. Otherwise this returns the result of the remote evaluation and
nil.
remote-value-bindis similar to
multiple-value-bindexcept the values bound come from
remote-form's evaluation in the remote process. The
local-formsexecute in an implicit
progn.
If the client unwinds past the call to
remote-value-bind, the server continues running, but the system ignores the values the server sends back.
If the server unwinds past the remotely requested call, instead of returning normally, the
local-formsnever execute, and
remote-value-bindreturns
nil.
The wire mechanism only directly supports a limited number of data types for transmission as arguments for remote function calls and as return values: integers inclusively less than 32 bits in length, symbols, lists. Sometimes it is useful to allow remote processes to refer to local data structures without allowing the remote process to operate on the data. We have
remote-objectsto support this without the need to represent the data structure in terms of the above data types, to send the representation to the remote process, to decode the representation, to later encode it again, and to send it back along the wire.
You can convert any Lisp object into a remote-object. When you send a remote-object along a wire, the system simply sends a unique token for it. In the remote process, the system looks up the token and returns a remote-object for the token. When the remote process needs to refer to the original Lisp object as an argument to a remote call back or as a return value, it uses the remote-object it has which the system converts to the unique token, sending that along the wire to the originating process. Upon receipt in the first process, the system converts the token back to the same (
eq) remote-object.
make-remote-objectreturns a remote-object that has
objectas its value. The remote-object can be passed across wires just like the directly supported wire data types.
The function
remote-object-preturns
tif
objectis a remote object and
nilotherwise.
The function
remote-object-local-preturns
tif
remoterefers to an object in the local process. This is can only occur if the local process created
remotewith
make-remote-object.
The function
remote-object-eqreturns
tif
obj1and
obj2refer to the same (
eq) lisp object, regardless of which process created the remote-objects.
This function returns the original object used to create the given remote object. It is an error if some other process originally created the remote-object.
This function removes the information and storage necessary to translate remote-objects back into
object, so the next
gccan reclaim the memory. You should use this when you no longer expect to receive references to
object. If some remote process does send a reference to
object,
remote-object-valuesignals an error.
The
wirepackage provides for sending data along wires. The
remotepackage sits on top of this package. All data sent with a given output routine must be read in the remote process with the complementary fetching routine. For example, if you send so a string with
wire-output-string, the remote process must know to use
wire-get-string. To avoid rigid data transfers and complicated code, the interface supports sending
taggeddata. With tagged data, the system sends a tag announcing the type of the next data, and the remote system takes care of fetching the appropriate type.
When using interfaces at the wire level instead of the RPC level, the remote process must read everything sent by these routines. If the remote process leaves any input on the wire, it will later mistake the data for an RPC request causing unknown lossage.
When using these routines both ends of the wire know exactly what types are coming and going and in what order. This data is restricted to the following types:
[Function]
wire:wire-get-byte wire[Function]
wire:wire-output-number wire number[Function]
wire:wire-get-number wire &optional signed[Function]
wire:wire-output-string wire string[Function]
wire:wire-get-string wireThese functions either output or input an object of the specified data type. When you use any of these output routines to send data across the wire, you must use the corresponding input routine interpret the data.
When using these routines, the system automatically transmits and interprets the tags for you, so both ends can figure out what kind of data transfers occur. Sending tagged data allows a greater variety of data types: integers inclusively less than 32 bits in length, symbols, lists, and
remote-objects(see section 9.1.3). The system sends symbols as two strings, the package name and the symbol name, and if the package doesn't exist remotely, the remote process signals an error. The system ignores other slots of symbols. Lists may be any tree of the above valid data types. To send other data types you must represent them in terms of these supported types. For example, you could use
prin1-to-stringlocally, send the string, and use
read-from-stringremotely.
[Function]
wire:wire-get-object wireThe function
wire-output-objectsends
objectover
wirepreceded by a tag indicating its type.
If
cache-itis non-
nil, this function only sends
objectthe first time it gets
object. Each end of the wire associates a token with
object, similar to remote-objects, allowing you to send the object more efficiently on successive transmissions.
cache-itdefaults to
tfor symbols and
nilfor other types. Since the RPC level requires function names, a high-level protocol based on a set of function calls saves time in sending the functions' names repeatedly.
The function
wire-get-objectreads the results of
wire-output-objectand returns that object.
You can create wires manually in addition to the
remotepackage's interface creating them for you. To create a wire, you need a Unix file descriptor. If you are unfamiliar with Unix file descriptors, see section 2 of the Unix manual pages.
The function
make-wirecreates a new wire when supplied with the file descriptor to use for the underlying I/O operations.
This function returns
tif
objectis indeed a wire,
nilotherwise.
This function returns the file descriptor used by the
wire.
The TCP/IP protocol allows users to send data asynchronously, otherwise known as
out-of-banddata. When using this feature, the operating system interrupts the receiving process if this process has chosen to be notified about out-of-band data. The receiver can grab this input without affecting any information currently queued on the socket. Therefore, you can use this without interfering with any current activity due to other wire and remote interfaces.
Unfortunately, most implementations of TCP/IP are broken, so use of out-of-band data is limited for safety reasons. You can only reliably send one character at a time.
The Wire package is built on top of CMUCLs networking support. In view of this, it is possible to use the routines described in section 10.6 for handling and sending out-of-band data. These all take a Unix file descriptor instead of a wire, but you can fetch a wire's file descriptor with
wire-fd.