Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow string operators to be disabled #80

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from

Conversation

bopjesvla
Copy link

@bopjesvla bopjesvla commented May 16, 2017

In Erlang's current state, it's impossible to impose memory limits on a process that allows string manipulation. max_heap_size does not help here, since large binaries are stored off-heap.

Most ways to manipulate or dynamically generate strings (string.*, tostring, and perhaps pcall) can already be disabled (#77). This PR introduces a way to disable string concatenation and other future string operators. For reference:

# limiting memory on heap

iex(4)> Process.flag(:max_heap_size, 30000)                                                                                                                                                                         
%{error_logger: true, kill: true, size: 0}                                                                                                                                                               
iex(5)> :luerl.eval("a={}; for i=1,1000000 do a[i] = 5 end")                                                                                                                 
** (EXIT from #PID<0.351.0>) killed                                                                                                                                                                          
                                                                                                                                                                                                             
Interactive Elixir (1.4.1) - press Ctrl+C to exit (type h() ENTER for help)                                                                                                                                  
iex(1)> [error]      Process:          #PID<0.351.0>                                                                                                                                                         
     Context:          maximum heap size reached                                                                                                                                                             
     Max Heap Size:    30000                                                                                                                                                                                 
     Total Heap Size:  96966                                                                                                                                                                                 
     Kill:             true                                                                                                                                                                                  
     Error Logger:     true                                                                                                                                                                                  
     GC Info:          [old_heap_block_size: 75113, heap_block_size: 21916, mbuf_size: 0,                                                                                                                    
 recent_size: 2643, stack_size: 60, old_heap_size: 55675, heap_size: 10895,                                                                                                                                  
 bin_vheap_size: 112, bin_vheap_block_size: 46422, bin_old_vheap_size: 596,                                                                                                                                  
 bin_old_vheap_block_size: 46422]             

# does not work on binaries 
                                                                                                                                                                                                             
iex(1)> Process.flag(:max_heap_size, 30000)                                                                                                                                                                  
%{error_logger: true, kill: true, size: 0}                                                                                                                                                                   
iex(2)> Application.put_env(:luerl, :no_string_ops, false)                                                                                                                                                         
:ok                                                                                                                                                                                                                                 
iex(3)> :luerl.eval("x = 'a'; while true do x = x .. x end")                                                                                                                                              
binary_alloc: Cannot reallocate 1073741855 bytes of memory (of type "binary").                                                                                                                               
                                                                                                                                                                                                             
Crash dump is being written to: erl_crash.dump...done                                                                           

# the solution                                                                                   
                                                                                                                                                                                                             
iex(1)> Process.flag(:max_heap_size, 30000)                                                                                                                                                                          
%{error_logger: true, kill: true, size: 0}                                                                                                                                                                   
iex(2)> Application.put_env(:luerl, :no_string_ops, true)                                                                                                                                                                    
:ok                                                                                                                                                                                                                                     
iex(3)> :luerl.eval("x = 'a' .. 'a'")                                                                                                                                                
{:error,                                                                                              
  {:lua_error, {:forbidden, :concat_op},
  ...
}

@bopjesvla
Copy link
Author

Fixed the example.

@bopjesvla
Copy link
Author

@rvirding, can you take a look at this? I'd like to give #78 a shot next week, but it would be nice to have some assurance that this project still takes pull requests.

@rvirding
Copy link
Owner

rvirding commented Oct 4, 2017

The idea is interesting but going out and checking the application environment for every concatenation call is a bit too slow. It also depends on the luerl application being loaded which is actually not needed now.

@bopjesvla
Copy link
Author

As I recall it, the reason I didn't use erlc macros was that I could not figure out how to configure them in Mix. If this is too slow, I could read the environment variable at compile time.

It also depends on the luerl application being loaded which is actually not needed now.

If the application is not loaded, get_env will return nil, so people will only have to configure the application if they want to use this specific feature.

I will likely get back to this PR in a month, when I might add the option to allow concatenations when the combined byte size does not exceed 64.

@bopjesvla
Copy link
Author

bopjesvla commented Apr 14, 2018

I modified the rebar script to also take configuration from the app env. I'd say this is good to go now.

@bopjesvla
Copy link
Author

Renamed NO_STRING_OPS to SANDBOX_STRINGS. When this flag is on, string literals and concatenated strings cannot have a byte size larger than 64.

* Store identifiers as strings rather than atoms at no penalty
* Limit identifier size to 64 characters if SANDBOX_STRINGS is set
@bopjesvla
Copy link
Author

Changed identifier storage from atoms to strings because they are only used as strings and dynamic atom creation is scary. ... still works. When SANDBOX_STRINGS is set, the length of identifiers is limited to 64.

yrashk added a commit to yrashk/bpxe.ex that referenced this pull request Dec 15, 2020
This seriously impedes further progress where logical gates (such as
inclusive/exclusive gateway) can be used.

Solution: execute scripts using Lua (luerl) as the first available language

Luerl is not perfect (but one can contribute to its development, I
guess?) and have some issues (see
rvirding/luerl#80 for a concerning example) but
that's the best one I can find that's *known* and works.
@rvirding rvirding force-pushed the develop branch 3 times, most recently from 4fdcb90 to c439c82 Compare February 26, 2022 20:58
@rvirding rvirding force-pushed the develop branch 2 times, most recently from 76b3c9b to 2124269 Compare January 28, 2024 01:14
@davydog187
Copy link
Collaborator

@bopjesvla do you still have an interest in solving this problem?

Since the PR is stale now, I will plan to close in a week or two unless I hear otherwise

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants