I need help to make AMT CIRA work for my public server which is running in hybrid mode. #6208

i-am-niru · 2024-06-27T06:57:22Z

i-am-niru
Jun 27, 2024

I am running the latest MeshCentral, which is in hybrid mode.

Server Setup:
Initially, on localhost (when I run MeshCentral on my laptop), everything works fine. Intel AMT CIRA is activated, and I can use the AMT functionality.

Issue with Remote Server:
We are running the MeshCentral server behind an Nginx ingress. Ports 443 and 4433 are accessible on the MeshCentral server. We are doing all of this with Kubernetes pod settings. All remote sessions, remote, backstage, and all functionalities work fine.

I am trying to activate and configure Intel AMT CIRA with CCM for my devices and need your help.

As we are using MeshCentral as part of our main web application, we have set MeshCentral on a subpath (route) of our main server. For example:

https://mainserver.com/remote/ (this will simply redirect to our login page)

We have written services in Kubernetes to forward this route with port to the MeshCentral pod/server but were unable to write a rule for 4433 due to its limitations.

So, we have created one public IP http://amtserver.com:4433 which redirects to meshcentral_server:4433 and it is working fine.

When we access http://amtserver.com:4433, it simply shows:
MeshCentral MPS server.
Intel® AMT computers should connect here.

In config.JSON, we are using mainserver.certificate in settings (using mainserver.com as a cert and certUrl) and set amtserver.com to mpsaliashost in the config.

json

{
  "$schema": "https://raw.githubusercontent.com/Ylianst/MeshCentral/master/meshcentral-config-schema.json",
  "settings": {
    "cert": "mainserver.com",
    "sessionSameSite": "lax",
    "port": 4430,
    "aliasPort": 443,
    "agentPort": 2121,
    "agentAliasPort": 443,
    "relayPort": 453,
    "agentPortTls": false,
    "agentPing": 10,
    "browserPing": 10,
    "agentIdleTimeout": 18000,
    "SelfUpdate": false,
    "allowFraming": true,
    "allowLoginToken": true,
    "webRTC": true,
    "mpsaliashost": "amtserver.com",
    "mpstlsoffload": true,
    "tlsoffload": true,
    "mpsport": 4433,
    "mpsaliasport": 4433,
    "plugins": {
      "enabled": true
    },
    "postgres": {
      "user": "admin",
      "password": "QqcqfrXq9CMRybnEekYd",
      "port": "5432",
      "host": "abc",
      "database": "meshremote",
      "ssl": false
    },
    "dbExpire": {
      "events": 0,
      "powerevents": 0,
      "statsevents": 0
    },
    "agentCoreDump": true,
    "agentLogDump": true,
    "minify": true,
    "clipboardGet": true,
    "clipboardSet": true,
    "compression": true,
    "wscompression": true,
    "agentwscompression": true,
    "noAgentUpdate": 1,
    "temporaryAgentUpdate": false,
    "agentsInRam": true,
    "desktopMultiplex": true
  },
  "domains": {
    "": {
      "title": "Remote server",
      "title2": "Remote server",
      "minify": "true",
      "NewAccounts": false,
      "certUrl": "mainserver.com",
      "notifyimage": abc.png",
      "meshMessengerTitle": "Messenger",
      "nightmode": 1,
      "userSessionIdleTimeout": 90,
      "allowedOrigin": true
    }
  }
}

Environment info

Node.js: v20.12.1
MeshCentral: 1.1.22
Nginx: We have Nginx before the MeshCentral server. We have also created a public IP [http://amtserver.com:4433] to redirect to MeshCentral:4433.
AMT Version: 11.0.18

AMT info
{
"core-ver": 1,
"OsHostname": "LAP-55",
"Flags": 4,
"MeiVersion": "100.0.0.5124",
"Versions": {
"Flash": "11.0.18",
"Netstack": "11.0.18",
"AMTApps": "11.0.18",
"AMT": "11.0.18",
"Sku": "16392",
"VendorID": "8086",
"Build Number": "1002",
"Recovery Version": "11.0.18",
"Recovery Build Num": "1002",
"Legacy Mode": "False"
},
"UUID": "MY UID",
"ProvisioningMode": 1,
"ProvisioningState": 2,
"net0": {
"enabled": 1,
"dhcpEnabled": 1,
"dhcpMode": "PASSIVE",
"mac": "D4:81:D7:A4:94:5A",
"address": "0.0.0.0"
},
"net1": {
"enabled": 1,
"dhcpEnabled": 1,
"dhcpMode": "PASSIVE",
"mac": "28:16:AD:8A:1E:1B",
"address": "0.0.0.0"
},
"DnsSuffix": "xyz.com"
}
Issue:
I am unable to activate Intel AMT and CIRA on my remote server. I am running the MeshCentral server with mpsdebug and getting the following logs in the console:

Logs under MyServer menu -> Trace tab -> CIRA SERVER AND CIRA COMMANDS CHECKED

Server Logs:
"mps","New CIRA connection"
"mps","CIRA connection closed"
"mps","New CIRA connection"
"mps","CIRA connection closed"
"mps","New CIRA connection"
"mps","CIRA connection closed"
"mps","New CIRA connection"
"mps","CIRA connection closed"
"mps","New CIRA connection"
"mps","CIRA connection closed"
"mps","New CIRA connection"
"mps","CIRA connection closed"
CIRA connection error: {"errno":-104,"code":"ECONNRESET","syscall":"read"}
"mps","CIRA connection closed"
"mps","New CIRA connection"
"mps","CIRA connection closed"
"mps","New CIRA connection"
"mps","CIRA connection closed"
"mps","New CIRA connection"
"mps","CIRA connection closed"
"mps","New CIRA connection"
"mps","CIRA connection closed"

Console Logs:
MPS <-- (5):6100000004
MPS --> (5):6100000004
MPS --> (55):5a0000000f666f727761726465642d74637069700000000500008000ffffffff000000000000426000000007312e322e332e3400000401
MPS <-- (17):5b000000050000000500008000ffffffff
MPS <-- (5):6100000005
MPS --> (5):6100000005
MPS --> (55):5a0000000f666f727761726465642d74637069700000000600008000ffffffff000000000000426000000007312e322e332e3400000402
MPS <-- (17):5b000000060000000600008000ffffffff
MPS <-- (5):6100000006
MPS --> (5):6100000006
MPS --> (55):5a0000000f666f727761726465642d74637069700000000700008000ffffffff000000000000426000000007312e322e332e3400000403
MPS <-- (17):5b000000070000000700008000ffffffff
MPS <-- (5):d0000000ff
MPS --> (5):d1000000ff
MPS <-- (5):6100000007
MPS --> (5):6100000007
MPS --> (55):5a0000000f666f727761726465642d74637069700000000800008000ffffffff000000000000426000000007312e322e332e3400000404
MPS <-- (17):5b000000080000000800008000ffffffff
MPS <-- (5):6100000008
MPS --> (5):6100000008
MPS --> (64):fa0000003b7b22616374696f6e223a22636f6e736f6c65222c226d7367223a224661696c656420746f2067657420496e74656c20414d542073746174652e227d
MPS --> (23):fa000000127b22616374696f6e223a22636c6f7365227d

I have some questions:

Do we need some certificate to activate AMT CIRA?
Do we need my machine in some specific network setup to activate CIRA? If yes, then what are those requirements?
How can I check if my machine is set up with the proper network and Intel BIOS settings?
What is the role of reverse proxy, DnsSuffix, certificates and DHCP ?

If this existing setup does not support CIRA, please suggest steps I can follow to check and resolve the issue.

si458 · 2024-06-27T07:37:02Z

si458
Jun 27, 2024
Collaborator

in your config.json, multitenant doesnt exist as a config option, so remove it
you have setup meshcentral on a subfolder on your reverse proxy https://mainserver.com/remote/
so you should make sure your reverse proxy REMOVES the /remote/ option because in the eyes of meshcentral /remote/ is a different domain and NOT your main "" blank domain!
you should be checking you can access your amt server using https://amtdomain.com:4433 and not http
you should be setting the dns suffix of your computer in the AMT to the same as your meshcentral server
you havent shared ur nginx config so the could be an issue with your nginx server

8 replies

i-am-niru Jul 4, 2024
Author

Hi,
As we are using nginx ingress controller which is hosted Azure Kubernetes service.
We tried to write nginx Ingress rule for https://amtdomain.com:4433 to make sure the remote devices can access https://amtdomain.com:4433 and it displays the meshcentral amt page but getting below error.
How to troubleshoot this error ?

Here is the nginx.config

server {
	server_name amtdomain.com ;
	
	listen 80  ;
	listen [::]:80  ;
	listen 443  ssl http2 ;
	listen [::]:443  ssl http2 ;
	
	set $proxy_upstream_name "-";
	
	ssl_certificate_by_lua_block {
		certificate.call()
	}
	
	location ~* "^/(/|$)(.*)" {
		
		set $namespace      "mesh-pod";
		set $ingress_name   "mesh-pod";
		set $service_name   "mesh-pod";
		set $service_port   "4433";
		set $location_path  "/(/|${literal_dollar})(.*)";
		set $global_rate_limit_exceeding n;
		
		rewrite_by_lua_block {
			lua_ingress.rewrite({
				force_ssl_redirect = false,
				ssl_redirect = true,
				force_no_ssl_redirect = false,
				preserve_trailing_slash = false,
				use_port_in_redirects = false,
				global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
			})
			balancer.rewrite()
			plugins.run()
		}
		
		# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
		# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
		# other authentication method such as basic auth or external auth useless - all requests will be allowed.
		#access_by_lua_block {
		#}
		
		header_filter_by_lua_block {
			lua_ingress.header()
			plugins.run()
		}
		
		body_filter_by_lua_block {
			plugins.run()
		}
		
		log_by_lua_block {
			balancer.log()
			
			monitor.call()
			
			plugins.run()
		}
		
		port_in_redirect off;
		
		set $balancer_ewma_score -1;
		set $proxy_upstream_name "mesh-pod-mesh-pod-4433";
		set $proxy_host          $proxy_upstream_name;
		set $pass_access_scheme  $scheme;
		
		set $pass_server_port    $server_port;
		
		set $best_http_host      $http_host;
		set $pass_port           $pass_server_port;
		
		set $proxy_alternative_upstream_name "";
		
		client_max_body_size                    5120m;
		
		proxy_set_header Host                   $best_http_host;
		
		# Pass the extracted client certificate to the backend
		
		# Allow websocket connections
		proxy_set_header                        Upgrade           $http_upgrade;
		
		proxy_set_header                        Connection        $connection_upgrade;
		
		proxy_set_header X-Request-ID           $req_id;
		proxy_set_header X-Real-IP              $remote_addr;
		
		proxy_set_header X-Forwarded-For        $remote_addr;
		
		proxy_set_header X-Forwarded-Host       $best_http_host;
		proxy_set_header X-Forwarded-Port       $pass_port;
		proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
		proxy_set_header X-Forwarded-Scheme     $pass_access_scheme;
		
		proxy_set_header X-Scheme               $pass_access_scheme;
		
		# Pass the original X-Forwarded-For
		proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
		
		# mitigate HTTPoxy Vulnerability
		# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
		proxy_set_header Proxy                  "";
		
		# Custom headers to proxied server
		
		proxy_connect_timeout                   5s;
		proxy_send_timeout                      60s;
		proxy_read_timeout                      60s;
		
		proxy_buffering                         off;
		proxy_buffer_size                       4k;
		proxy_buffers                           4 4k;
		
		proxy_max_temp_file_size                1024m;
		
		proxy_request_buffering                 off;
		proxy_http_version                      1.1;
		
		proxy_cookie_domain                     off;
		proxy_cookie_path                       off;
		
		# In case of errors try the next upstream server before returning an error
		proxy_next_upstream                     error timeout;
		proxy_next_upstream_timeout             0;
		proxy_next_upstream_tries               3;
		
		rewrite "(?i)/(/|$)(.*)" /$2 break;
		proxy_pass http://upstream_balancer;
		
		proxy_redirect                          off;
		
	}
	
	location ~* "^/" {
		
		set $namespace      "mesh-pod";
		set $ingress_name   "mesh-pod";
		set $service_name   "";
		set $service_port   "";
		set $location_path  "/";
		set $global_rate_limit_exceeding n;
		
		rewrite_by_lua_block {
			lua_ingress.rewrite({
				force_ssl_redirect = false,
				ssl_redirect = true,
				force_no_ssl_redirect = false,
				preserve_trailing_slash = false,
				use_port_in_redirects = false,
				global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
			})
			balancer.rewrite()
			plugins.run()
		}
		
		# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
		# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
		# other authentication method such as basic auth or external auth useless - all requests will be allowed.
		#access_by_lua_block {
		#}
		
		header_filter_by_lua_block {
			lua_ingress.header()
			plugins.run()
		}
		
		body_filter_by_lua_block {
			plugins.run()
		}
		
		log_by_lua_block {
			balancer.log()
			
			monitor.call()
			
			plugins.run()
		}
		
		port_in_redirect off;
		
		set $balancer_ewma_score -1;
		set $proxy_upstream_name "upstream-default-backend";
		set $proxy_host          $proxy_upstream_name;
		set $pass_access_scheme  $scheme;
		
		set $pass_server_port    $server_port;
		
		set $best_http_host      $http_host;
		set $pass_port           $pass_server_port;
		
		set $proxy_alternative_upstream_name "";
		
		client_max_body_size                    5120m;
		
		proxy_set_header Host                   $best_http_host;
		
		# Pass the extracted client certificate to the backend
		
		# Allow websocket connections
		proxy_set_header                        Upgrade           $http_upgrade;
		
		proxy_set_header                        Connection        $connection_upgrade;
		
		proxy_set_header X-Request-ID           $req_id;
		proxy_set_header X-Real-IP              $remote_addr;
		
		proxy_set_header X-Forwarded-For        $remote_addr;
		
		proxy_set_header X-Forwarded-Host       $best_http_host;
		proxy_set_header X-Forwarded-Port       $pass_port;
		proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
		proxy_set_header X-Forwarded-Scheme     $pass_access_scheme;
		
		proxy_set_header X-Scheme               $pass_access_scheme;
		
		# Pass the original X-Forwarded-For
		proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
		
		# mitigate HTTPoxy Vulnerability
		# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
		proxy_set_header Proxy                  "";
		
		# Custom headers to proxied server
		
		proxy_connect_timeout                   5s;
		proxy_send_timeout                      60s;
		proxy_read_timeout                      60s;
		
		proxy_buffering                         off;
		proxy_buffer_size                       4k;
		proxy_buffers                           4 4k;
		
		proxy_max_temp_file_size                1024m;
		
		proxy_request_buffering                 off;
		proxy_http_version                      1.1;
		
		proxy_cookie_domain                     off;
		proxy_cookie_path                       off;
		
		# In case of errors try the next upstream server before returning an error
		proxy_next_upstream                     error timeout;
		proxy_next_upstream_timeout             0;
		proxy_next_upstream_tries               3;
		
		rewrite "(?i)/" /$2 break;
		proxy_pass http://upstream_balancer;
		
		proxy_redirect                          off;
		
	}
}

Please tell me if i missed to understand some part.

si458 Jul 4, 2024
Collaborator

i dont think you have setup ur nginx correctly, you have setup only 80/443 (listen 80 and 443 lines)

so im guessing the port 4433 must be hitting a different service in ur kubernetes cluster and not ur nginx or even ur meshcentral server

you know IF your amt is working correctly because you should get in your web browser
MeshCentral MPS server. Intel® AMT computers should connect here.

your picture doesnt show that message so something is wrong with your setup

if you are happy to pay for support, and you can give me access, i can look into it for you,
just email me (my gthub page for email address)

i-am-niru Aug 1, 2024
Author

Hi @si458

Thanks for the reply and suggestions.

I tried some approaches, and here is an update.

Now, Intel AMT is activated in ACM mode with tls , and the AMT server at https://amtdomain.com:4433 is accessible from the browser, displaying a success message as well.
MeshCentral MPS server is where Intel® AMT computers should connect.

The issue was due to the '/remote/' path. After the request, it was trying to fetch the '/remote/' key from obj.httpsresponses, but only '/' was present. so I added /remote/ and it works

obj.httpResponses = {
    '/remote/': '<!DOCTYPE html><html><head><meta charset="UTF-8"></head><body>MeshCentral MPS server.<br />Intel&reg; AMT computers should connect here.</body></html>'
    //'/text.ico': { file: 'c:\\temp\\test.iso', maxserve: 3, maxtime: Date.now() + 15000 }
};

However, CIRA is still not getting activated. I think the connection request is coming from the client and reaching the AMT MPS server. The request is accepted but then disconnects/closes immediately.

I also checked the remote server for AMT info using meshcmd.

C:\Users\Admin\Desktop>meshcmd.exe amtinfo
Intel AMT v11.8.93, activated in Admin Control Mode (ACM).
Wired Enabled, DHCP, D4:81:D7:A4:94:5A
Wireless Enabled, DHCP, 28:16:AD:8A:1E:1B
Connection Status: Outside, CIRA: Connecting to amtserver.com, Periodic.

I checked the logs in the mpsserver.js file. Below are the logs from the onConnection function, where we are handling all incoming requests, I guess.

AMT Test CIRA: CreateMpsServer -> connectionCount 
Inside parent debug source ---> mps
Inside parent debug args ---> [ 'New CIRA connection' ]
Inside parent debug source ---> mps
Inside parent debug args ---> [ 'CIRA connection closed' ]
mps CIRA connection closed
Connection closed
Had error: false
Bytes read: 0
Bytes written: 0
Remote address: ::ffff:12.114.0.44
Remote port: 6220
Local address: undefined
Local port: undefined

I am getting the above logs repeatedly. It appears that the connection is being accepted but is immediately closed with no data being read or written.

So,

Is my remote system network configuration settings correct, or am I missing something causing it to disconnect immediately?
Could there be a subpath-related issue (/remote/) causing the server to be unable to process the request/response?
Also, the local address and port are undefined. Is this okay?

si458 Aug 1, 2024
Collaborator

I'm not sure WHY you are adding /remote/ ?
You shouldn't be using ANY SUB FOLDERS AT ALL !?
port 4433 should be directly connected to the meshcentral machine, if that's not possible and u are using a reverseproxy, then proxy port 4433 straight through to the meshcentral server!
Don't edit anyting, don't change anything, just a straight through

i-am-niru Aug 1, 2024
Author

Okay. I removed the /remote/ from the code.
Now , able to access https://amtserver.com:4433 from the browser
but still CIRA is not connected. Connecting to amtserver.com, Periodic.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

I need help to make AMT CIRA work for my public server which is running in hybrid mode. #6208

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment 8 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

I need help to make AMT CIRA work for my public server which is running in hybrid mode. #6208

i-am-niru Jun 27, 2024

Replies: 1 comment · 8 replies

si458 Jun 27, 2024 Collaborator

i-am-niru Jul 4, 2024 Author

si458 Jul 4, 2024 Collaborator

i-am-niru Aug 1, 2024 Author

si458 Aug 1, 2024 Collaborator

i-am-niru Aug 1, 2024 Author

i-am-niru
Jun 27, 2024

Replies: 1 comment 8 replies

si458
Jun 27, 2024
Collaborator

i-am-niru Jul 4, 2024
Author

si458 Jul 4, 2024
Collaborator

i-am-niru Aug 1, 2024
Author

si458 Aug 1, 2024
Collaborator

i-am-niru Aug 1, 2024
Author