Tauri: Much smaller download size and reduced resource consumption

[terreng]

I'm wondering if it's possible to run the servers from a separate node process, and kill the electron process when the window isn't visible. I'm not sure how much energy is used to run in the background. Separating it like this would likely make it open slower.

I want to look in to possibly using Tauri instead of Electron for a future version. Depends how well it works and how mature it is. Benefit would be much smaller package size and memory overhead.

[terreng]

Tauri plans to have its first stable release in Q1 2022, and support for Deno is on the roadmap. I'm going to keep my eyes on this, because not needing to bundle Chromium with the app would majorly save on package size and resource utilization.

[terreng]

Revisit #74 (Compression)

[terreng]

https://tauri.studio/v1/guides/features/system-tray/

tauri-apps/tauri#2684

[terreng]

@ethanaobrien I've reached the conclusion that Tauri simply isn't ready for our project yet. It's missing a few features we need, so we'll have to wait for them to be added. They've also suggested that other programming languages may be supported in the future, including Python, so it seems better to wait and see what happens before rewriting the program in Rust. I'll revisit this, maybe in a year, but for now we're better off sticking with Node and Electron.

[ethanaobrien]

Sounds good,

including Python

You're not suggesting we use python, are you? Python is much, much slower than javascript. I found a video on a speed comparison here (and the guy actually uses nodejs)

[terreng]

I wasn't thinking about it in terms of speed, although that's certainly important. The Tauri website mentions "Go, Nim, Python, C++." C++ might be a better choice.

[ethanaobrien]

C++ might be a better choice.

I completely agree. It's not too big in size at all and about as efficient as you can get. Although most of the apis are different per system. Would this be a problem?

[terreng]

I'm not sure because support for these languages hasn't been released yet. We'll just have to wait and see.

[ethanaobrien]

@terreng I had another idea the other day and wanted your opinion on it. I don't know if you've heard of it but there's this program called unity that's commonly known for their game editor. I think this might be a good idea for another lightweight web server translation. The downside is if we were to use the unity is the splash screen, but that's not too bad. The average size wouldn't be too big 50-150mb and it's 100% cross platform (including android, macos, windows, Linux, ios, etc...) and wouldn't be a bad idea. It uses C# in the backend and even has something similar to css, although ive never messed with that feature. I think we should have this electron version as a more advanced version and we could make another version in something like unity for people looking for a simpler/faster option. I know you'd like to stick to something that supports html/css as the frontend, but if we truly want to have both size and to be widely supported I don't think this is a realistic goal. A more "lite" version wouldn't need nearly as many options either. I ask you look into my suggestions and seriously think about them. Is look or performance better?

[terreng]

@ethanaobrien Thanks for sharing the idea. I have used Unity before. I think that using a game engine for any purpose other than to make a game is generally a mistake. It's a mistake that I've made before, and I don't intend to make that mistake again. These game engines are simply not designed to make general purpose software like this program. There are many better alternatives for our use case.

something like unity for people looking for a simpler/faster option

I don't believe that Unity would offer either a simpler or a faster alternative to Electron.

I believe some of the advantages you listed are advantages that Tauri would offer.

it's 100% cross platform (including android, macos, windows, Linux, ios, etc...)

Tauri plans to support mobile in the future.

I know you'd like to stick to something that supports html/css as the frontend, but if we truly want to have both size and to be widely supported I don't think this is a realistic goal.

The goal seems fairly realistic with Tauri. I'm not adamant about having an interface made with HTML/CSS, but at the moment it genuinely seems like the most reasonable cross-platform option.

Just to clarify, I am thinking about a long term solution. Tauri is still in development of course, but looks to be backed by some big companies.

To me, the best path forward looks like sticking with Electron + Node.js until Tauri is in a place where we can use it, and then switching over. We may be able to switch over without much of a loss of functionality, but we can still offer the Electron version if necessary.

I do genuinely appreciate the suggestion, and I welcome any other suggestions you might have. I'm not dead set on Tauri. That said, please trust that I'm confident Unity is not the right direction for this project.

[ethanaobrien]

Hey @terreng, I wanted to let you know I've been working on writing a basic web server in rust that we can use for the Tauri build. It should be done by now actually. The only parts I have left are parts where I actually process and respond to the request.

Heres the project so far: https://github.com/ethanaobrien/Rust-server

Intended to somewhat look like nodejs (calling read_string with 0 reads all available data)

Update on the current progress of the server:

• Range handling
• Exported (fully working) url decode function
• Keep alive connections
• Chunked encoding
• Ability to consume request body
• Set content type based on extension

Also wanted to note, when the user does not set the content length (just like it does in nodejs) chunked encoding is automatically enabled. The server itself is almost finished

[terreng]

Hey Ethan, this is really great, well done! I saw your message two days ago and I appreciate the edit & update, sorry for the delay in my response. I think this means that I should work out the Tauri stuff, right? I think I already have a branch with the UI ported to Tauri, so in theory we should be able to merge in your web server code once it's done and have a working app.

[ethanaobrien]

I think this means that I should work out the Tauri stuff, right?

Yeah, that'd be helpful. Here's what I'm thinking, could you get the UI and the basic backend setup while I finish refining the server? What I'm thinking is supplying a settings struct to a function that creates a server. (I'll be adding a server.kill() method for termination).

Something like this:

struct Settings {
    port: i32,
    path: &str,
    /* ... */
}

fn start_server(config: Settings) -> Server {
    /* I put server code here */
}

While I think in the future we can make .swshtaccess server side scripts and plugins work, its not in my plans right now.

Also I had a thought, should we rename the .swshtaccess feature, since in reality its completely a completely different feature and format. What do you think about this, especially with the re-write.

I think I already have a branch with the UI ported to Tauri, so in theory we should be able to merge in your web server code once it's done and have a working app.

Could you get me a struct of settings that'd you'd like in the first update, (possibly ones we will do later), and I'll get right on writing those options. The backend server should be completely finished, I just need to know what I need to do from here to make sure it works under every case.

[terreng]

Yeah, that'd be helpful. Here's what I'm thinking, could you get the UI and the basic backend setup while I finish refining the server?

I should have time in the next few days to revisit the tauri branch and see what I have left to do. I think Tauri has also had some major updates since then so I might need to review that.

What do you think about this, especially with the re-write.

I think this gives us an opportunity to re-think the way advanced configuration works. I haven't looked in to running arbitrary JavaScript from within the Rust code, but it should be possible (I think we could create a hidden WebView and run the script through that, although it might not be very performant). So we could theoretically create a JavaScript plugin system just like in the old version. We could also consider some other scripting language. PHP comes to mind. Maybe that's all too complicated and we should do something like the . swshtaccess feature under a new name / format.

Either way, I think that decision can come after an initial Tauri version.

Could you get me a struct of settings that'd you'd like in the first update

I'm thinking we should do all of them, except htaccess as you mentioned. If you don't have the bandwidth for that, are you asking me to prioritize which options are most important?

Also, this might be a good opportunity to revise some of the options we have. In particular, I think we could do away with ipThrottling and maybe customErrorReplaceString. Perhaps we should even rethink having custom error pages at all, since it's really not necessary.

Maybe we could have a better custom-header system, where you can specify the value for any header, rather than having dedicated cacheControl and cors options. We could still have buttons in the UI for common headers.

Let me know your thoughts on these. Maybe the next step is to create a new, revised list of options.

[ethanaobrien]

I think we could create a hidden WebView and run the script through that, although it might not be very performant

This is what I was thinking.

We could also consider some other scripting language. PHP comes to mind.

That... uhh, goes back to the same reason it didn't in the first place.

EDIT: apparently this is possible? (http://phpjs.hertzen.com/) but would still require using javascript and would probably be less performant

I'm thinking we should do all of them, except htaccess as you mentioned. If you don't have the bandwidth for that, are you asking me to prioritize which options are most important?

That should be manageable. If I got the time I may even work on the htaccess feature (what should we rename it to?) because I myself need the ability to set custom headers (because of this, and note sending this header with all files could be unsafe). Not really asking what's important, was more asking if there are any options you want to take out.

Also, this might be a good opportunity to revise some of the options we have. In particular, I think we could do away with ipThrottling and maybe customErrorReplaceString.

I agree with getting rid of the ipThrottling and customErrorReplaceString options.

Perhaps we should even rethink having custom error pages at all, since it's really not necessary.

My view on this is it isn't really difficult to just send an alternate html file instead of a default 404 - File not found message. (and most if not all other web servers support this option)

Maybe we could have a better custom-header system, where you can specify the value for any header, rather than having dedicated cacheControl and cors options. We could still have buttons in the UI for common headers.

In my opinion, I think we should at least have a cors option (since there seem to be a lot of people that don't actually know what the error cors is), but other than that a field to modify any header would be great.

Also, do we have a list of options anywhere

Does this look good?

struct Settings {
    port: i32,
    path: &str,
    local_network: bool,
    spa: bool,
    rewrite_to: &str,
    directory_listing: bool,
    exclude_dot_html: bool,
    ipv6: bool,
    hidden_dot_files: bool,
    cors: bool,
    upload: bool,
    replace: bool,
    delete: bool,
    static_directory_listing: bool,
    hidden_dot_files_directory_listing: bool,
    custom404: &str,
    custom403: &str,
    custom401: &str,
    http_auth: bool,
    http_auth_username: &str,
    http_auth_password: &str,
}

The one option I may not be able to do at this moment is https.

[terreng]

Options look good except for static_directory_listing. Do we really need that?

[ethanaobrien]

Options look good except for static_directory_listing. Do we really need that?

In my experience, yeah. On older browsers, for some reason, the normal directory listing doesnt work at all.

[terreng]

We should be able to fix that problem. What's the issue? Is there a script error or something?

The specifics are beside the point, actually. The point is that we should be able to fix the root cause of that issue, obviating the need for the option.

[ethanaobrien]

Did some debugging, Its a script error. I should be able to fix it, and catch it and render the static listing at the very least.

[ethanaobrien]

Alright @terreng I've got communication with the server done. It might be a little hard to work with, but I couldn't do it any other way.

main.rs shows how to start the server, see if the server started, and kill the server. If you need anything else, please let me know, but after several hours of trying to figure it out, this was the only way I could get it to work so I'll see whether or not I can manage it.

[terreng]

Nice! I looked in to Tauri a bit today and there are still that I need to figure out. One might be a PR for the Tauri project because it's missing security scoped bookmarks. So there's definitely a decent amount of work left to do on that side.

[ethanaobrien]

@terreng Wanted to let you know that the part youd need to communicate with has been finalized. I'm onto the handling part now. Its much cleaner than I had initially resulted with (see here).

Example:

let mut server = SimpleWebServer::new(settings);
let started = server.start();
println!("Server started: {}", started);
//server.terminate(); to terminate the server

[terreng]

Great!

[ethanaobrien]

@terreng

The backend is done. You can run it from this repo: https://github.com/ethanaobrien/Rust-server

Has the following options implimented:

pub struct Settings<'a> {
    pub port: i32,
    pub path: &'a str,
    pub local_network: bool,
    pub spa: bool,
    pub rewrite_to: &'a str,
    pub directory_listing: bool,
    pub exclude_dot_html: bool,
    pub ipv6: bool,
    pub hidden_dot_files: bool,
    pub cors: bool,
    pub upload: bool,
    pub replace: bool,
    pub delete: bool,
    pub hidden_dot_files_directory_listing: bool,
    pub custom401: &'a str,
    pub custom403: &'a str,
    pub custom404: &'a str,
    pub custom500: &'a str,
    pub http_auth: bool,
    pub http_auth_username: &'a str,
    pub http_auth_password: &'a str,
    pub index: bool
}

I might start to take a look writing the htaccess feature if I have time, speaking of that, I need your opinion on 2 things.

1. What should the new feature name be called? I'd like to determine this before I start writing the backend to avoid confusion on what features are in the backend.
2. What should the data format for the new feature be since JSON wouldn't be very easy to parse?

[terreng]

The backend is done. You can run it from this repo:

That's fantastic! Could you remind me why the https option was hard to implement?

I might start to take a look writing the htaccess feature if I have time, speaking of that, I need your opinion on 2 things.

Off the top of my head, I don't know the answer for either of these. In this situation I would probably look around at other web servers and see what they call it and what format they use.

[ethanaobrien]

That's fantastic! Could you remind me why the https option was hard to implement?

Thats right, I forgot https. This will definitely take priority. Its going to be slightly difficult because I dont know how tls encryption works, and both Web server for chrome and the electron version of this project use libraries to do it. I'm hoping theres a library to do this in rust too.

[terreng]

I'm hoping theres a library to do this in rust too.

There certainly is.

[ethanaobrien]

There certainly is.

Would you mind sending the link to the crate?

[terreng]

I asked Google Bard, here's what it gave me:

AI generated answer

To rewrite the Node.js code you provided in Rust, you can use the following crates:

• webpki: This crate provides a library for working with X.509 certificates and other PKI-related data.
• ring: This crate provides a low-level cryptographic library that implements common cryptographic algorithms, such as AES, RSA, and SHA-2.

The following code shows how to use these crates to rewrite the Node.js code in Rust:

use webpki::*;
use ring::{
    rand::SystemRandom,
    signature::{RsaKeyPair, RsaSigningAlgorithm},
};

fn generate_self_signed_certificate() -> Certificate {
    let mut rng = SystemRandom::new();
    let key_pair = RsaKeyPair::generate_pkcs8(&rng, 1024).unwrap();
    let mut cert = Certificate::new();
    cert.set_serial_number(0);
    cert.set_validity(Validity::new(chrono::Utc::now(), chrono::Utc::now() + chrono::Duration::years(10)));
    cert.set_subject_name(SubjectName::new(
        vec![
            Rdn::CommonName("WebServerForChrome".to_string()),
            Rdn::CountryName("US".to_string()),
            Rdn::StateOrProvinceName("test-st".to_string()),
            Rdn::LocalityName("Simple Web Server".to_string()),
            Rdn::OrganizationName("Simple Web Server".to_string()),
            Rdn::OrganizationalUnitName("WSC".to_string()),
        ],
    ));
    cert.set_issuer_name(cert.get_subject_name());
    cert.set_basic_constraints(BasicConstraints::new(true));
    cert.set_key_usage(KeyUsage::new(
        true,
        true,
        true,
        true,
        true,
    ));
    cert.set_subject_alternative_names(vec![
        GeneralName::Uri("http://localhost".to_string()),
    ]);
    cert.set_public_key(key_pair.public_key().as_ref());
    cert.sign(
        &key_pair,
        &RsaSigningAlgorithm::SHA256,
    )
    .unwrap();
    cert
}

fn main() {
    let cert = generate_self_signed_certificate();
    let cert_pem = cert.to_pem().unwrap();
    let private_key_pem = key_pair.private_key().to_pem().unwrap();

    // Save the certificate and private key to a file
    // ...

    println!("Certificate created for \"WebServerForChrome\": \n");
    println!("{}", cert_pem);
}

This code will generate a self-signed X.509 certificate with the subject name "WebServerForChrome". The certificate will be valid for 10 years and will have the following extensions:

• Basic constraints: CA = true
• Key usage: digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
• Subject alternative names: http://localhost

The certificate and private key will be saved to a file, which can then be used to secure a web server.

[ethanaobrien]

Added https support @terreng

ethanaobrien/Rust-server@e8fd168

[terreng]

That's awesome! I can see that it's totally up to me now to figure out the Tauri issues. The big one is the lack of security scoped bookmarks on macos. I'll check if there are any other crates we could use to work around this limitation.

[ethanaobrien]

If it would help, I can go ahead and make a bookmark.rs file that, with dummy get and release functions for the server backend, and call these in the background so all we need to do is actually implement the feature

[terreng]

Sure yeah, that's a good idea.

[ethanaobrien]

@terreng Forgot to tell you the basic features (by basic I mean everything but the advanced features) are added.

Have you tried it yet? I could build you a binary to have you test it if you dont have the ability to do it yourself. I have tested and it works 100% on windows, linux, and macos. (I have basic command line arguments support)

[terreng]

Thanks for the update! I have yet to try it out. I believe you that it works. The next step is definitely to hook it up to the UI with Tauri, which is when I would test it. I'm a little busy with things at the moment so I won't be able to do this until next month.

[terreng]

So sorry I haven't had the chance to work on this. The front end and new rust back end are both done, all that's missing is the glue to bring them together (Tauri). But not having any Rust experience myself, this last piece of the puzzle is a time commitment that I haven't been able to fit in yet.

[ethanaobrien]

It's alright, I've been quite busy too