help request: ext-plugin(java) error with multipart/form-data? #282

channer99 · 2023-12-22T01:36:10Z

Description

apisix-java-plugin-runner version is 0.4.0

request Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryedcKdnGDWD6rJDcu

payload is
------WebKitFormBoundaryedcKdnGDWD6rJDcu
Content-Disposition: form-data; name="file"; filename="test.xlsx"
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet

------WebKitFormBoundaryedcKdnGDWD6rJDcu
Content-Disposition: form-data; name="age"

20
------WebKitFormBoundaryedcKdnGDWD6rJDcu--

but ext 503 error occured..

error.log is below.. is that library issue?

pls check this log

2023/12/22 09:46:12 [warn] 52461#52461: *63 [lua] init.lua:956: 2023-12-22 09:46:12.590 ERROR 52473 --- [tLoopGroup-34-4] o.a.a.p.r.h.RpcCallHandler               : handle request error:

java.lang.IndexOutOfBoundsException: null
        at java.nio.Buffer.checkIndex(Buffer.java:687) ~[?:?]
        at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:269) ~[?:?]
        at io.github.api7.A6.ExtraInfo.Resp.result(Resp.java:18) ~[A6-0.6.0-RELEASE.jar!/:?]
        at org.apache.apisix.plugin.runner.ExtraInfoResponse.getResult(ExtraInfoResponse.java:41) ~[apisix-runner-plugin-sdk-0.4.0.jar!/:0.4.0]
        at org.apache.apisix.plugin.runner.handler.RpcCallHandler.handleExtraInfo(RpcCallHandler.java:218) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
        at org.apache.apisix.plugin.runner.handler.RpcCallHandler.channelRead0(RpcCallHandler.java:92) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
        at org.apache.apisix.plugin.runner.handler.RpcCallHandler.channelRead0(RpcCallHandler.java:54) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
        at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at org.apache.apisix.plugin.runner.handler.PrepareConfHandler.channelRead0(PrepareConfHandler.java:57) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
        at org.apache.apisix.plugin.runner.handler.PrepareConfHandler.channelRead0(PrepareConfHandler.java:45) ~[apisix-runner-core-0.4.0.jar!/:0.4.0]
        at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
2023/12/22 09:46:12 [warn] 52461#52461: *63 [lua] init.lua:956: y.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) ~[netty-handler-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[netty-transport-classes-epoll-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.epoll.EpollDomainSocketChannel$EpollDomainUnsafe.epollInReady(EpollDomainSocketChannel.java:138) ~[netty-transport-classes-epoll-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:487) ~[netty-transport-classes-epoll-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385) ~[netty-transport-classes-epoll-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.78.Final.jar!/:4.1.78.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.78.Final.jar!/:4.1.78.Final]
        at java.lang.Thread.run(Thread.java:829) ~[?:?]

, context: ngx.timer
2023/12/22 09:46:12 [warn] 52461#52461: *63 [lua] init.lua:956: 2023-12-22 09:46:12.597  WARN 52473 --- [tLoopGroup-34-4] o.a.a.p.r.h.PayloadDecoder               : receive unsupported type: -4
, context: ngx.timer
2023/12/22 09:46:12 [warn] 52461#52461: *63 [lua] init.lua:956: 2023-12-22 09:46:12.605  WARN 52473 --- [tLoopGroup-34-4] o.a.a.p.r.h.PayloadDecoder               : receive unsupported type: -72
, context: ngx.timer
2023/12/22 09:46:12 [error] 52455#52455: *6011569 [lua] init.lua:884: phase_func(): failed to receive RPC_HTTP_REQ_CALL: service unavailable, client: 
2023/12/22 09:46:12 [warn] 52455#52455: *6011569 [lua] plugin.lua:1102: run_plugin(): ext-plugin-pre-req exits with http status code 503,

Environment

APISIX version (run apisix version): 3.4.1
Operating system (run uname -a):
OpenResty / Nginx version (run openresty -V or nginx -V):
etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
APISIX Dashboard version, if relevant:
Plugin runner version, for issues related to plugin runners:
LuaRocks version, for installation issues (run luarocks --version):

The text was updated successfully, but these errors were encountered:

sheharyaar · 2023-12-22T03:45:08Z

Hi, @channer99 , can you pleasee send the following in correct format :

Routes, services and other APISIX configs used
The complete steps to reproduce the issue.

Please format the configs and logs using backticks(`)

channer99 · 2023-12-22T06:30:54Z

@sheharyaar

route config is

{
  "uri": "/test*",
  "name": "test",
  "methods": [
    "GET",
    "POST",
    "PUT",
    "DELETE",
    "PATCH",
    "OPTIONS"
  ],
  "plugins": {
    "ext-plugin-pre-req": {
      "_meta": {
        "disable": false
      },
      "allow_degradation": false,
      "conf": [
        {
          "name": "Test_filter",
          "value": "{\"rejected_body2\":\"failed\",\"rejected_code2\":\"403\"}"
        }
      ]
    },
  },
  "upstream_id": "473424305712006806",
  "status": 1
}`

apisix config.yaml

apisix:
  node_listen:
      - port: 80
  ssl:
    enable: false

deployment:
  role: traditional
  role_traditional:
    config_provider: etcd
  admin:
    allow_admin:
      - all
    admin_listen:
        port: 9180
    admin_key:
      - name: admin
        key: edd1c9f034335f136f87ad84b625c8f1  # using fixed API token has security risk, please update it when you deploy to production environment
        role: admin
  etcd:
    host:
      - "http://xxx.xx.xxx.xxx:2379"

nginx_config:
  enable_cpu_affinity: |
      true;
  http_configuration_snippet: |
      client_body_buffer_size 10m;
      proxy_max_temp_file_size 0;
  http_server_configuration_snippet: |
      merge_slashes off;
      proxy_ignore_client_abort on;

ext-plugin:
  cmd: ['java', '-jar', '-Xmx1g', '-Xms1g', '/usr/local/apisix/test_filter.jar']

channer99 · 2023-12-22T07:01:33Z

attached test.xlsx is 30MB excel.

my request code(java spring) is

String filePath = "/Users/aaaa/Desktop/test.xlsx";
        Path path = Paths.get(filePath);
        byte[] fileData = null;
        try {
            fileData = Files.readAllBytes(path);
        } catch (IOException e) {
            e.printStackTrace();
        }

        String url2 = "/test";

        HttpHeaders headers2 = new HttpHeaders();
        headers2.setContentType(MediaType.MULTIPART_FORM_DATA);
        headers2.set("boundary", "----WebKitFormBoundarykMBT8G8fZJ6YOU6X");

        MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
        body.add("file", new HttpEntity<>(fileData, getMultipartFileHeaders(path.getFileName().toString())));

        HttpEntity<MultiValueMap<String, Object>> entity2 = new HttpEntity<>(body, headers2);

        RestTemplate restTemplate2 = new RestTemplate();
        restTemplate2.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
        ResponseEntity<String> response2 = restTemplate2.exchange("http://{gw}" + url2, HttpMethod.POST,
                entity2, String.class);

If you refer to the above, you will be able to reproduce it easily.
503 occurs before ext filter logic is performed.

pls check @sheharyaar

sheharyaar · 2023-12-22T07:50:37Z

Ok, I will check and update you soon. I cannot find the attached xls, I will create a dummy xls for this.

channer99 · 2023-12-22T08:03:49Z

Ok, I will check and update you soon. I cannot find the attached xls, I will create a dummy xls for this.

@sheharyaar
It seems to be a file size issue. When I reduced the size of the same xlsx file to 10mb, the call was successful. Is it not possible to call large files larger than 10MB in apisix?

sheharyaar · 2023-12-22T08:45:46Z

Ca you check the timeout for this error?? Maybe try setting larger timeout : https://apisix.apache.org/docs/apisix/admin-api/

channer99 · 2023-12-26T00:13:04Z

increased all timeout(connect, read, send), but it doesn't work

shreemaan-abhishek transferred this issue from apache/apisix Dec 22, 2023

shreemaan-abhishek removed this from Apache APISIX backlog Dec 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

help request: ext-plugin(java) error with multipart/form-data? #282

help request: ext-plugin(java) error with multipart/form-data? #282

channer99 commented Dec 22, 2023 •

edited

Loading

sheharyaar commented Dec 22, 2023 •

edited

Loading

channer99 commented Dec 22, 2023 •

edited

Loading

channer99 commented Dec 22, 2023 •

edited

Loading

sheharyaar commented Dec 22, 2023

channer99 commented Dec 22, 2023 •

edited

Loading

sheharyaar commented Dec 22, 2023

channer99 commented Dec 26, 2023

help request: ext-plugin(java) error with multipart/form-data? #282

help request: ext-plugin(java) error with multipart/form-data? #282

Comments

channer99 commented Dec 22, 2023 • edited Loading

Description

Environment

sheharyaar commented Dec 22, 2023 • edited Loading

channer99 commented Dec 22, 2023 • edited Loading

channer99 commented Dec 22, 2023 • edited Loading

sheharyaar commented Dec 22, 2023

channer99 commented Dec 22, 2023 • edited Loading

sheharyaar commented Dec 22, 2023

channer99 commented Dec 26, 2023

channer99 commented Dec 22, 2023 •

edited

Loading

sheharyaar commented Dec 22, 2023 •

edited

Loading

channer99 commented Dec 22, 2023 •

edited

Loading

channer99 commented Dec 22, 2023 •

edited

Loading

channer99 commented Dec 22, 2023 •

edited

Loading