Try on VM
This guide will walk you through a basic Camblet scenario. It will deploy Camblet into the kernel to assign strong identities to processes and transparently establish mTLS connections. Simple NGINX server with cURL will be used for demostration.
In this scenario there will be a NGINX server which will work as an echo server. On client side cURL will be used. All commands must be run inside the virtual machine. To return to Lima:
limactl shell quickstart
Install NGINX server:
sudo apt install nginx -y
Configure agent
Camblet consist of two building blocks:
- Kernel module: Handles transparent TLS and enforces policies.
- Agent: Signs certificates and collects metadata for processes.
The agent configuration resides in /etc/camblet/config.yaml
.
By default, it looks like this:
agent: trustDomain: acme.corp defaultCertTTL: 2h metadataCollectors: procfs: enabled: true extractEnvs: false linuxos: enabled: true sysfsdmi: enabled: true azure: enabled: false ec2: enabled: false gcp: enabled: false kubernetes: enabled: false docker: enabled: false
In this file, the agent can be configured to use a trust domain of your choice, along with the certificate time-to-live (certTTL).
Camblet support various metadata sources, those can be configured under the metadataCollectors
block.
Camblet can utilize metadata from these sources to identify processes and enforce policies. The procfs, linuxos, and sysfsdmi are enabled by default.
Let’s gather metadata for the NGINX web server that was just installed.
camblet agent augment $(pidof -s nginx)
linuxos:kernel:release:linux-6.5.0-14-genericlinuxos:kernel:version:#14-Ubuntu SMP PREEMPT_DYNAMIC Tue Nov 14 14:59:49 UTC 2023linuxos:name:ubuntulinuxos:version:Ubuntu 23.10process:cmdline:nginx: worker processprocess:gid:33process:gid:additional:33process:gid:effective:33process:gid:real:33process:name:nginxprocess:pid:3731process:uid:33process:uid:effective:33process:uid:real:33sysfsdmi:bios:date:03/01/2023sysfsdmi:bios:release:0.0sysfsdmi:bios:vendor:EDK IIsysfsdmi:bios:version:edk2-stable202302-for-qemusysfsdmi:chassis:type:1sysfsdmi:chassis:vendor:QEMUsysfsdmi:chassis:version:pc-q35-8.2sysfsdmi:product:name:Standard PC (Q35 + ICH9, 2009)sysfsdmi:product:version:pc-q35-8.2
The command prints out every available metadata for the specified process. These metadata can be used in policies to describe identities and related parameters.
Create policy for NGINX
The policy can be written by hand, but the Camblet CLI can do the work for you with the generate-policy
command. There are two mandatory parameters, the PID of the process and the workload ID.
Let’s run the Camblet CLI to generate a policy and save it into the default policy directory as nginx.yaml.
sudo camblet agent generate-policy $(pidof -s nginx) nginx | sudo tee /etc/camblet/policies/nginx.yaml
- certificate: ttl: 86400s workloadID: nginx connection: mtls: STRICT selectors: - process:binary:path: /usr/sbin/nginx process:gid: "33" process:name: nginx process:uid: "33"
Camblet will use the selectors
to identify the NGINX server. The connection
part configures the TLS settings where STRICT
mTLS value means only clients with trusted certificates can communicate with it.
To verify that, let’s try connecting to NGINX with cURL.
curl localhost
curl: (56) Recv failure: Connection reset by peer
The connection failed, as it was supposed to. Since Camblet now protects NGINX with mTLS, so nothing can communicate with it without a trusted client certificate.
The protection can be simply checked by openssl s_client
.
openssl s_client -connect localhost:80 2>/dev/null
CONNECTED(00000003)---Certificate chain 0 s:CN = camblet-protected-workload i:CN = Camblet root CA a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256 v:NotBefore: Feb 21 13:44:20 2024 GMT; NotAfter: Feb 22 13:44:20 2024 GMT---Server certificate-----BEGIN CERTIFICATE-----MIIDSDCCAjCgAwIBAgIUGBEwCjEpqr+6IuiCLtk6TLIh2oIwDQYJKoZIhvcNAQELBQAwGjEYMBYGA1UEAxMPQ2FtYmxldCByb290IENBMB4XDTI0MDIyMTEzNDQyMFoXDTI0MDIyMjEzNDQyMFowJTEjMCEGA1UEAxMaY2FtYmxldC1wcm90ZWN0ZWQtd29ya2xvYWQwggEgMA0GCSqGSIb3DQEBAQUAA4IBDQAwggEIAoIBAQDCgZnRjiiSlrAZQqOr0C4o+2uWKqPQdFENNqp3qZAMkq6tSYlkEyf6q9KPFI+V9gOxwPkAFPamizTH3d57974oBmNK2BwtYiyhLus0AnAVahSYgqefCJ7EyYXw62Ip93Bqjdvu+P4KLLHe2Vdy/tT3T05aQTn9yguvhdQjIncUoPleU+UFFatv+/qkBUKQnr284ado+YQvTD2UjCL/s/yhyhbZ0M3NugZn+dZuqDRB7dt6/rZOQ/McPJ0FzxZDK53dWz6xQb+5yGHmYbJKNcGtPyPycKjhnmcEzowBT4L+uTnVPxSr8kP8KqzspfIBRLPC4vURyYwl9to9fJBkY7AFAgEDo30wezAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMB8GA1UdIwQYMBaAFKGmr3y8dJsskFwuowJaVVzBru2kMCkGA1UdEQQiMCCGHnNwaWZmZTovL2FjbWUuY29ycC9lY2hvLXNlcnZlcjANBgkqhkiG9w0BAQsFAAOCAQEAFm5svzOUQAMR79d+MbCOHtrixGzipPCuIa2locGo7vLk/mdkPYSU1dQsR0pCBpXuSfuTIhLG1WvSN6weIHisVhikWKKq0WCw02MwhyyZoh0E3PXu3iE+nY2w6RUCAR6Ok2goH0bw6vVpmHklFJl4XHSzmzVbvNwqg3IL1+lR2cf67FeoN3Wmi0wD0Pe0E5Z5ObQ1uHaxOvoDl4OqBfu7M5HOyq1eUcxWyj15Zbp/aUq/Uil05YYSndYJ49nDAki5jWVjXPlnehtvVV0pHtSzuOcXDlOaBEpuAiadO6/gqPQFm4++0uQsSZHkbGhwqHCfsBVudroOusDXAKvDIuLmwA==-----END CERTIFICATE-----subject=CN = camblet-protected-workloadissuer=CN = Camblet root CA---Acceptable client certificate CA namesCN = Camblet root CAClient Certificate Types: RSA signRequested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA512:RSA+SHA224:RSA+SHA1Shared Requested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA512:RSA+SHA224Peer signing digest: SHA256Peer signature type: RSAServer Temp Key: X25519, 253 bits---SSL handshake has read 1294 bytes and written 398 bytesVerification error: unable to verify the first certificate---New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384Server public key is 2048 bitSecure Renegotiation IS supportedNo ALPN negotiatedSSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES256-GCM-SHA384 Session-ID: 64217D9B0EE7BE52F229F9D30BD7949C3494019E957A76C9385F60C146030F09 Session-ID-ctx: Master-Key: 0C1F47031A5C1408C3B85D09F492FAC3086112F5A8AA800145AB1E49163A446A09CD56F7EC59EF195453E1E158EB0A25 PSK identity: None PSK identity hint: None SRP username: None Start Time: 1708523445 Timeout : 7200 (sec) Verify return code: 21 (unable to verify the first certificate) Extended master secret: no---
Create policy for cURL
To create a new policy for cURL to communicate with the server, we need the PID of cURL. Since cURL is not running continuously like the server, a “dummy” command is needed to force it to run until the policy is generated.
curl 1.2.3.4 &
Let’s generate a policy for cURL
This must be done before cURL times out and the process terminates.
sudo camblet agent generate-policy $(pidof curl) curl | sudo tee /etc/camblet/policies/curl-simple.yaml
- certificate: ttl: 86400s workloadID: curl connection: mtls: STRICT selectors: - process:binary:path: /usr/bin/curl process:gid: "1000" process:name: curl process:uid: "501"
Try to connect to NGINX with a certificate
Using this policy, cURL gets an identity and will use mTLS. Let’s try to communicate with the NGINX once again.
curl localhost
curl: (56) Recv failure: Connection reset by peer
It still doesn’t work, one last piece of the puzzle is missing. Camblet must ascertain the target destination for the application of policies. Enforcing policies on every egress connection implies that cURL won’t be able to reach destinations beyond the Camblet-managed environment. Services has to be registered into the service registry through service definitions.
Sample Service discovery configuration file
- addresses: - address: localhost port: 80 labels: app:label: app
Let’s ignore the labels part for now; it is meant for more advanced configuration.
The addresses
section specifies the registered destination addresses to which the policies should be applied.
Let’s create our own services.yaml
where we place the nginx server’s service IP and port number.
sudo tee /etc/camblet/services/local.yaml >/dev/null <<EOF- addresses: - address: 127.0.0.1 port: 80 labels: app:label: nginxEOF
Try to connect to NGINX with a certificate and configured Camblet
With the service registry entry for NGINX in place, let’s check if its indeed fixed the error.
curl localhost
<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style>html { color-scheme: light dark; }body { width: 35em; margin: 0 auto;font-family: Tahoma, Verdana, Arial, sans-serif; }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>
<p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p></body></html>
It finally works.
Cleanup
Exit from the Lima VM and delete it to cleanup.
limactl delete quickstart --force