The @web module contains functions for interfacing with the World Wide Web.

Functions

:req(url,data,headers,method) #

Make a web request.

Parameters

Return

Notes

The headers User-Agent, Accept, and Content-Length are automatically included in the request.

Any additional request headers can be specified via headers, which should be a case-sensitive map of header name to header value.

If method is empty and data is null then method becomes 'GET'. If data is not null then method becomes 'POST'.

The server's SSL certificate is not validated. You can check the certificate yourself against a known certificate or authority chain if validation is desired.

The function :get_header() can be used to parse out a single header value from the raw response headers.

Example

res = web.req('http://example.com/')
print res.head
##
HTTP/1.1 200 OK
Age: 509983
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
...
##
print res.body
##
<!doctype html>
<html>
<head>
    <title>Example Domain</title>
...
##
:dl(url,max_age) #

Download a file to the Axiom cache.

Parameters

Return

Notes

This uses the same cache as :import(), which is the folder .axiom.

This will show a progress bar while the file is downloading.

The the URL is already cached and the cache value is younger than max age, then no download occurs. (The cached file path is just returned.)

On error, the path is returned but the file will be empty.

:get(url,max_age) #

Get a URL with caching. Like :dl(), but returns response body instead of path of cached file.

Parameters

Return

Notes

This is equivalent to: io.load(web.dl(url))

Like :dl(), a progress bar is shown when a request actually occurs (cache miss).

:get_header(name,head) #

Parse out the value of a single header from a string of raw headers.

Parameters

Return

:get_cookies(head) #

Get cookies from raw response headers.

Parameters

Return

:submit(url,form,headers) #

Do a form submission

Parameters

Return

Notes

This is equivalent to doing a form submit from an HTML form like: 

<form action="url" method="POST">
	<input name="key0" value="value0" />
	<input name="key1" value="value1" />
	...
</form>

Where keyN and valueN are key/value pairs in form.

To submit mutiple form fields with the same name, the value of the form field should be an array.

To submit files, the value of the form field should be an object with the following fields:

:url(s) #

Split a URL into parts.

Parameters

Return

Notes

If the "host" contains Unicode, it will be converted into an Internationalized Domain Name with Punycode.

If no protocol is specified, "http" is assumed. If the URL is protocol-relative (begins with "//"), "https" is assumed.

Example

print web.url('https://example.com/whatever?q=something')
##{
  "protocol" : "https",
  "host" : "example.com",
  "port" : 443,
  "resource" : "/whatever?q=something"
}##

print web.url('//example.com/whatever?q=something')
##{
  "protocol" : "https",
  "host" : "example.com",
  "port" : 443,
  "resource" : "/whatever?q=something"
}##

print web.url('δΎ‹.com/whatever?q=something')
##{
  "protocol" : "http",
  "host" : "xn--fsq.com",
  "port" : 80,
  "resource" : "/whatever?q=something"
}##
:user_agent(name) #

Get or set user agent name.

Parameters

Return

Notes

The user agent is sent in all requests as part of the request header.

The default user agent is "Mozilla/5.0" for maximum compatibility.

Example

print web.user_agent() #show current user agent
#mimic Python
web.user_agent('Python-urllib/2.6')
#mimic IE9, the most popular browser in 2011
web.user_agent('Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)')
#mimic Android Chrome, the most popular browser in 2023
web.user_agent('Mozilla/5.0 (Linux; Android 13; Pixel 6a) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Mobile Safari/537.36')
:content_type(filepath) #

Get the HTTP content type for a given file. The content type is inferred from the extension, or the first 128 bytes if none. See notes for details.

Parameters

Return

Notes

The following extensions are supported:

If the filepath has no extension or the extension is not supported above, the first 128 bytes are inspected to see if they contain a null byte. If so, "application/x-octet-stream" is returned, otherwise "text/plain" is returned.

@server Functions

:use(on_request,on_upgrade,on_message,on_destroy) #

Set the global web server event handlers.

Parameters

Notes

All of the handlers take no parameters, and should use the other web.server functions to get context information.

By default (when the handlers are null) the on_request/on_message handlers echo back the request/message (for debugging), and on_upgrade closes the connection immediately (to deny unwanted websocket connections).

Example

#this is code for a simple chat server
:on_request()
	if web.server.path()=='/'
		web.server.write_file('chat.html')
	else
		web.server.write_404()

:on_upgrade()
	if web.server.path()=='/chat'
		web.server.subscribe('chat')
		this.name = randstr()
		web.server.publish('chat','['+this.name+' has entered the chat room]',1)
	else
		web.server.close()

:on_message()
	if web.server.path()=='/chat'
		web.server.publish('chat','['+this.name+']: '+web.server.message(),1)
	else
		web.server.close()

:on_destroy()
	if web.server.path()=='/chat'
		web.server.publish('chat','['+this.name+' has left the chat room]',1)

web.server.use(on_request,on_upgrade,on_message,on_destroy)
web.server.run({
	port: 8999
})

See chat.html for the corresponding HTML/CSS/JS code.

:run(config) #

Run the global web server using specified configuration.

Parameters

Notes

The default config file (web.ini) looks like this: 

port = <random integer in range [1000,9999]>
local = true
secret = <random UUID>
;crt = server.crt
;key = server.key

Instead of a config file, an object can be passed containing all the necessary variables. At a bare minimum, the "port" variable is required, although it is recommended to specify "local" and "secret" as well.

The global web server handles two routes automatically:

Any route of the form "/static/*" will serve the corresponding file in the subdirectory named "static". Simply do not create a subdirectory named "static" if you do not want this behavior.

Any route of the form "/web/*" will serve the corresponding file in the directory specified by the environment variable "WEB", or the subdirectory named "web" if the environment variable does not exist. This feature is to facilitate sharing of client-side code between projects during development (via environment variable), while also allowing for deployments (via subdirectory). If the environment variable "WEB" is specified, it should contain an absolute path with no trailing slash. Simply do not create an environment variable named "WEB" and do not create a subfolder named "web" if you do not want this behavior.

This function blocks until the global web server is shut down (by ctrl-c or process signal/kill). This function should only be called from the main thread. Consider using background threads or processes if background processing is needed.

Example

#create a file called web.ini if not exist, and use those values
#do nothing except serve files from /static/ and /web/
web.server.run()
:method() #

Get the current request method.

Return

:resource() #

Get the current request resource.

Return

Notes

The resource is the path plus any querystring. Use :path() to get just the path. Use :querystring() to get just the querystring. Unlike :path(), the :resource() is not modified by :pre() and :pop().

:path() #

Get the current request path.

Return

Notes

Excludes the querystring. Use :querystring() to get the querystring.

:headers() #

Get the raw headers for the current request.

Return

:content() #

Get the current request content (body).

Return

:querystring() #

Get the current request querystring.

Return

Notes

Excludes leading question mark. For example if the full path was "/index.html?abc=123", the querystring would be "abc=123".

Use :load_get() to parse the querystring.

:load_get(delimiter) #

Parse getdata from querystring.

Parameters

Return

:load_post() #

Parse postdata from request body.

Return

Notes

Multiple form fields with the same name are combined into a single array value.

Files become objects with the following fields:

:load_auth() #

Load and unscramble basic authorization header if specified

Return

Notes

Use :write_401() to force a login prompt in browsers.

:client_ip() #

Get the client IPv4 as a string.

Return

Notes

If the client IP is 127.0.0.1 (localhost), this function will look for the IP in the "X-Forwarded-For" header, in order to get the true client IP when behind a reverse proxy.

:pre(prefix) #

Checks if path matches prefix. If so, returns true and removes prefix from path.

Parameters

Return

Notes

This is used for routing. When returning true, path is modified, so subsequent calls to :path will be return different results.

Example

#map routes like "/abc/*" to files in subdirectory "xyz"
:on_request()
	if web.server.pre('/abc/')
		web.server.write_file('xyz/'+web.server.path())
	else
		web.server.write_404()

web.server.use(on_request)
web.server.run()
:pop() #

Pop leftmost token off path until next slash. This modifies the path.

Return

Notes

This is used for routing. Tokens are defined as the strings in the path delimited by forward slashes. If there are no more tokens to pop, return empty string and also set path to empty string. (For example, if path is "/" and pop is called, path becomes "" and "" is returned.)

Example

#handle route like /thing/[id]
:on_request()
	if web.server.pre('/thing/')
		id = web.server.pop().number()
		#...
		web.server.write_json({id:id,data:'whatever'})

web.server.use(on_request)
web.server.run()
:get_header(name) #

Get header from current request by name.

Parameters

Return

:get_cookie(name) #

Get cookie from current request by name.

Parameters

Return

:set_cookie(name,value,domain) #

Set a response cookie.

Parameters

Notes

The domain flag is used to share cookies across subdomains. For example, if login.example.com wants to share cookies with api.example.com, set domain to true.

:wipe_cookie(name) #

Erase a response cookie from client.

Parameters

Notes

This tells the client to erase the cookie on its end by setting Max-Age to 0.

:get_signed_cookie(name,max_age) #

Get cookie from current request by name.

Parameters

Return

Notes

This uses the "secret" specified as part of the configuration for :run().

:set_signed_cookie(name,value,domain) #

Set a signed response cookie.

Parameters

Notes

The domain flag is used to share cookies across subdomains. For example, if login.example.com wants to share cookies with api.example.com, set domain to true.

This uses the "secret" specified as part of the configuration for :run().

:redirect(url,permanent) #

Write a redirect response.

Parameters

Notes

If permanent is true, the response status is "301 Moved Permanently". If permanent is false, the response status is "302 Found".

:write(response_headers,body) #

Write a response.

Parameters

Notes

This function provides low-level access for writing generic responses. It is recommended to use the other web.server.write_ functions for convenience.

The response_headers field should contain the status line followed by the headers. Each line in response_headers should be followed by "\r\n". However, the entire string should NOT contain a trailing "\r\n". (So response_headers end with "\r\n" not "\r\n\r\n".)

The "Content-Length" header is automatically added if not already present in response_headers.

The body is automatically ignored if responding to a HEAD request.

:write_200(content_type,data) #

Write a 200 OK response.

Parameters

:write_401(realm) #

Write a 401 Unauthorized response. This causes the browser to prompt for Basic Authorization.

Parameters

Notes

Although the specs say "401 Unauthorized", most other servers return "401 Not Authorized", so this function does the same.

:write_403() #

Write a 403 Forbidden response.

:write_404() #

Write a 404 Not Found response.

:write_405(methods) #

Write a 405 Method Not Allowed response.

Parameters

:write_500() #

Write a 500 Internal Server Error response.

:write_file(path,content_type) #

Write a 200 OK response using a file.

Parameters

Notes

If content_type is blank, it is automatically chosen based on the file extension in path.

:write_json(data,cors) #

Write a 200 OK response with type "application/json" using any variable.

Parameters

Notes

The data parameter can be a variable of any type, and is automatically converted to JSON for the response.

:write_thread(fn,self,args,content_type) #

Write a 200 OK response after running a function in a separate thread.

Parameters

Notes

Use this function for long-running responses so that they do not block the main thread (and other connections) from processing.

This operates by creating a new thread, then calling fn with self and args in the new thread. After the function completes, the return value of the function is used as the response body, and content_type is used as the Content-Type header.

This function sets the timeout of the current connection to -1 (unlimited). To reduce this, use :timeout().

:timeout(secs) #

Get or set the timeout for the current connection.

Parameters

Return

Notes

The default timeout for all connections is 10 seconds. The timeout becomes -1 (unlimited) if :write_thread() is called, or the connection is upgraded to a websocket. In both cases, this function can be used to reduce the timeout.

:minify_js(path) #

Minify Javascript file and return minified contents.

Parameters

Return

Notes

This function just removes comments and whitespace. It does not rename function or variable names.

:minify_css(path) #

Minify CSS file and return minified contents.

Parameters

Return

Notes

This function just removes comments and whitespace. It does not rename any selectors.

:render_html(path,context) #

Render an HTML template and return the rendered contents.

Parameters

Return

Notes

This function uses an extremely simple template syntax, where $variable is replaced with context[variable]. Use $$ to escape $. The variable $this becomes the filename from path without the extension, to aid with scoping.

The template syntax also supports server-side includes in the following format:

<!--#include path.html title="Title" name="value"-->

Where "path.html" is the path of the file to include (also treated like a template), and any attributes are used to form the context for the sub-template.

:html_escape(s) #

HTML-escape a string.

Parameters

Return

Notes

This simply replaces '<', '>', '&', and '"' with their corresponding HTML entities.

:close() #

Close the websocket.

:message() #

Get the current websocket message from the client to the server.

Return

:send_message(msg,opcode) #

Send a message on the current websocket from server to client.

Parameters

Notes

If opcode is 1, the msg becomes a String within the browser. If opcode is 2, the msg becomes a Blob within the browser.

:subscribe(channel) #

Subscribe the current websocket to channel. This causes messages :publish() to the channel to be relayed to websocket client.

Parameters

:publish(channel,msg,opcode) #

Publish a message to the specified websocket channel.

Parameters

Notes

If opcode is 1, the msg becomes a String within the browser. If opcode is 2, the msg becomes a Blob within the browser.