X11's fundamental concept is that of the server and client. The server is the program which manages the graphics display, and clients are programs which connect to the server and (usually) request graphics to be displayed. X11 uses a protocol, the X Protocol that allows communication between the server and clients to be encapsulated in nearly any type of network protocol that provides a reliable two-way byte stream (eg TCP, Chaosnet, DECnet, Unix sockets). As such X is network-transparent, and clients can execute on host computers separate from the computer which hosts the server.
The X Protocol is language-independent. The primary implementation is in a weird object-oriented-ish form of ANSI C, but the X Protocol can be implemented in nearly any programming language. As such, Common Lisp has its own implementation of the X Protocol, called CLX.
MIT has long since let go of X, and it is now officially maintained and standardized by X.Org (formerly by the MIT X Consortium, then by The Open Group). Many different implementations of X exist, each typically supported by a particular Unix vendor. However the original source code for X was and continues to be freely available.