Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: How to setup scheduled task with manual cron input #1522

Closed
clementchong opened this issue Jun 27, 2022 · 26 comments
Closed

Question: How to setup scheduled task with manual cron input #1522

clementchong opened this issue Jun 27, 2022 · 26 comments
Assignees
Labels
improvement New feature or request status: volunteer wanted Need more input from community.

Comments

@clementchong
Copy link
Contributor

clementchong commented Jun 27, 2022

Hello, I tried to setup a weekly schedule but realized only possible with manual input. I like to ask what value should the question mark take when using manual setup? As per cron syntax, that value should be the path to the script we want to trigger.

This is not specified in the user guide document.

image

@clementchong
Copy link
Contributor Author

Existing codes has week options logic but is not provided as an option on frontend. Is it feature removed or left out by mistake?

image

@scottsut scottsut added the improvement New feature or request label Jun 27, 2022
@scottsut
Copy link
Contributor

We left out week option for some reason, it will be added in a later version.

@clementchong
Copy link
Contributor Author

For manual input, do we leave the "?" as is? Meaning only specify the first 5 values for cron timing.

@scottsut
Copy link
Contributor

No, the "?" is also part of the cron expression.

@clementchong
Copy link
Contributor Author

Yes, what should we replace the "?" with? If we use the normal day, month, year options, there is no need to enter this value.

@scottsut
Copy link
Contributor

It seems that there are several different implementations of cron expressions. Datart uses quartz type.

@clementchong
Copy link
Contributor Author

Thank you, it looks like we can leave the "?" as is.

This issue is tagged for volunteer, do you mean to add the week option?

@scottsut
Copy link
Contributor

Thank you, it looks like we can leave the "?" as is.

This issue is tagged for volunteer, do you mean to add the week option?

Yes.

@clementchong
Copy link
Contributor Author

I would try

@clementchong
Copy link
Contributor Author

clementchong commented Jun 28, 2022

Somehow the existing scheduled task stopped working. My smtp is tested working. Datart scheduled task was working before and I have not changed it. What could be the reason?

Even the demo site scheduled task feature is not working.

I like to make sure it is working before I make any change for testing.

image

image

@clementchong
Copy link
Contributor Author

smtp online test is successful and test mail received. Not working from datart yet.

image

@scottsut
Copy link
Contributor

scottsut commented Jun 29, 2022

It should be noted that the password configuration does not refer to the mailbox password, but the authorization code (may be this name, each mailbox is different)

image

@clementchong
Copy link
Contributor Author

Yes, has been using Google app password. Is not mailbox password.

@scottsut
Copy link
Contributor

Anything details in log file?

@clementchong
Copy link
Contributor Author

2022-06-13 13:54:43.251 [https-jsse-nio-0.0.0.0-9293-exec-8] INFO datart.server.service.impl.ScheduleServiceImpl : schedule started.
2022-06-13 13:54:43.255 [https-jsse-nio-0.0.0.0-9293-exec-8] INFO datart.server.service.impl.ScheduleServiceImpl : Executing job [Weekly Summary,job type :EMAIL]
2022-06-13 13:54:50.180 [https-jsse-nio-0.0.0.0-9293-exec-8] ERROR datart.server.job.ScheduleJob : schedule execute error
javax.script.ScriptException: ReferenceError: "process" is not defined in at line number 16483
at jdk.nashorn.api.scripting.NashornScriptEngine.throwAsScriptException(NashornScriptEngine.java:470)
at jdk.nashorn.api.scripting.NashornScriptEngine.invokeImpl(NashornScriptEngine.java:392)
at jdk.nashorn.api.scripting.NashornScriptEngine.invokeFunction(NashornScriptEngine.java:190)
at datart.server.common.JsParserUtils.parseExecuteParam(JsParserUtils.java:21)
at datart.server.job.ScheduleJob.doGetData(ScheduleJob.java:133)
at datart.server.job.ScheduleJob.execute(ScheduleJob.java:100)
at datart.server.service.impl.ScheduleServiceImpl.execute(ScheduleServiceImpl.java:198)
at datart.server.service.impl.ScheduleServiceImpl$$FastClassBySpringCGLIB$$a6988452.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
at datart.server.service.impl.ScheduleServiceImpl$$EnhancerBySpringCGLIB$$97f16206.execute()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy186.execute(Unknown Source)
at datart.server.controller.ScheduleController.execute(ScheduleController.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:197)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:141)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:894)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1060)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:962)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:652)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:887)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1684)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Unknown Source)
Caused by: jdk.nashorn.internal.runtime.ECMAException: ReferenceError: "process" is not defined
at jdk.nashorn.internal.runtime.ECMAErrors.error(ECMAErrors.java:57)
at jdk.nashorn.internal.runtime.ECMAErrors.referenceError(ECMAErrors.java:319)
at jdk.nashorn.internal.runtime.ECMAErrors.referenceError(ECMAErrors.java:291)
at jdk.nashorn.internal.objects.Global.noSuchProperty(Global.java:1442)
at jdk.nashorn.internal.scripts.Script$Recompilation$545$644853IA$^eval_.L:5#n(:16483)
at jdk.nashorn.internal.scripts.Script$Recompilation$544$646842A$^eval_.L:5#b(:16483)
at jdk.nashorn.internal.scripts.Script$Recompilation$543$648338AAA$^eval_.L:5#R(:16483)
at jdk.nashorn.internal.scripts.Script$Recompilation$539$653179AAA$^eval_.L:5#un#e#produce(:16483)
at jdk.nashorn.internal.scripts.Script$Recompilation$538$843036AA$^eval_.L:5#updateBy(:21453)
at jdk.nashorn.internal.scripts.Script$Recompilation$537$849342A$^eval_.L:5#getRuntimeDateLevelFields(:21625)
at jdk.nashorn.internal.scripts.Script$Recompilation$533$857300AADA$^eval_.L:5#ChartDataRequestBuilder#buildGroups#groupColumns(:21790)
at jdk.nashorn.internal.objects.NativeArray$12.forEach(NativeArray.java:1699)
at jdk.nashorn.internal.runtime.arrays.IteratorAction.apply(IteratorAction.java:110)
at jdk.nashorn.internal.objects.NativeArray.reduceInner(NativeArray.java:1702)
at jdk.nashorn.internal.objects.NativeArray.reduce(NativeArray.java:1714)
at jdk.nashorn.internal.scripts.Script$Recompilation$532$857004$^eval_.L:5#ChartDataRequestBuilder#buildGroups(:21781)
at jdk.nashorn.internal.scripts.Script$Recompilation$519$867422$^eval_.L:5#ChartDataRequestBuilder#build(:21960)
at jdk.nashorn.internal.scripts.Script$Recompilation$517$912302A$^eval_.L:5#getChartQueryData(:23040)
at jdk.nashorn.internal.scripts.Script$Recompilation$516$913157AA$^eval_.L:5#getQueryData(:23049)
at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:641)
at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494)
at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393)
at jdk.nashorn.api.scripting.ScriptObjectMirror.callMember(ScriptObjectMirror.java:199)
at jdk.nashorn.api.scripting.NashornScriptEngine.invokeImpl(NashornScriptEngine.java:386)
... 99 common frames omitted
2022-06-13 13:54:50.194 [https-jsse-nio-0.0.0.0-9293-exec-8] INFO datart.server.service.impl.ScheduleServiceImpl : job execute finished with failure

@scottsut
Copy link
Contributor

Somehow the existing scheduled task stopped working. My smtp is tested working. Datart scheduled task was working before and I have not changed it. What could be the reason?

Even the demo site scheduled task feature is not working.

I like to make sure it is working before I make any change for testing.

image

image

mail needs to be under spring.

@clementchong
Copy link
Contributor Author

You have sharp eyes! Let me get the email sending working.

@clementchong
Copy link
Contributor Author

Email sending works after changing from port 587 to 465. Are we missing any configuration parameters for port 587 to work?

image

@clementchong
Copy link
Contributor Author

clementchong commented Jun 30, 2022

I realized my scheduler does not work, is there any other configuration required? I see the below mentioned NOT STARTED in my log. I run via "Execute Now" and email works.

2022-06-30 22:23:01.973 [main] INFO org.quartz.core.SchedulerSignalerImpl : Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
2022-06-30 22:23:01.973 [main] INFO org.quartz.core.QuartzScheduler : Quartz Scheduler v.2.3.2 created.
2022-06-30 22:23:01.977 [main] INFO o.s.scheduling.quartz.LocalDataSourceJobStore : Using db table-based data access locking (synchronization).
2022-06-30 22:23:01.980 [main] INFO o.s.scheduling.quartz.LocalDataSourceJobStore : JobStoreCMT initialized.
2022-06-30 22:23:01.981 [main] INFO org.quartz.core.QuartzScheduler : Scheduler meta-data: Quartz Scheduler (v2.3.2) 'DatartScheduleCluster' with instanceId 'ubuntu1656598981939'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.springframework.scheduling.quartz.LocalDataSourceJobStore' - which supports persistence. and is clustered.

2022-06-30 22:23:01.981 [main] INFO org.quartz.impl.StdSchedulerFactory : Quartz scheduler 'DatartScheduleCluster' initialized from an externally provided properties instance.
2022-06-30 22:23:01.981 [main] INFO org.quartz.impl.StdSchedulerFactory : Quartz scheduler version: 2.3.2

@clementchong
Copy link
Contributor Author

clementchong commented Jun 30, 2022

I realized my scheduler does not work, is there any other configuration required? I see the below mentioned NOT STARTED in my log. I run via "Execute Now" and email works.

2022-06-30 22:23:01.973 [main] INFO org.quartz.core.SchedulerSignalerImpl : Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl 2022-06-30 22:23:01.973 [main] INFO org.quartz.core.QuartzScheduler : Quartz Scheduler v.2.3.2 created. 2022-06-30 22:23:01.977 [main] INFO o.s.scheduling.quartz.LocalDataSourceJobStore : Using db table-based data access locking (synchronization). 2022-06-30 22:23:01.980 [main] INFO o.s.scheduling.quartz.LocalDataSourceJobStore : JobStoreCMT initialized. 2022-06-30 22:23:01.981 [main] INFO org.quartz.core.QuartzScheduler : Scheduler meta-data: Quartz Scheduler (v2.3.2) 'DatartScheduleCluster' with instanceId 'ubuntu1656598981939' Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally. NOT STARTED. Currently in standby mode. Number of jobs executed: 0 Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads. Using job-store 'org.springframework.scheduling.quartz.LocalDataSourceJobStore' - which supports persistence. and is clustered.

2022-06-30 22:23:01.981 [main] INFO org.quartz.impl.StdSchedulerFactory : Quartz scheduler 'DatartScheduleCluster' initialized from an externally provided properties instance. 2022-06-30 22:23:01.981 [main] INFO org.quartz.impl.StdSchedulerFactory : Quartz scheduler version: 2.3.2

I see that there is a start button for each scheduled task. Now the scheduled task get triggered automatically as expected. However, there is a SQL error on column 'type'. Not sure if this is existing issue. I tried doing a fresh installation of datart database and the view table does not have type column.

image

image

Full log:

Error querying database. Cause: java.sql.SQLSyntaxErrorException: Unknown column 'type' in 'field list' ### The error may exist in datart/core/mappers/ext/ViewMapperExt.java (best guess) ### The error may involve datart.core.mappers.ext.ViewMapperExt.selectByPrimaryKey-Inline ### The error occurred while setting parameters ### SQL: select id, name, description, org_id, source_id, script, type, model, config, create_by, create_time, update_by, update_time, parent_id, is_folder, index, status from view where id = ? ### Cause: java.sql.SQLSyntaxErrorException: Unknown column 'type' in 'field list' ; bad SQL grammar []; nested exception is java.sql.SQLSyntaxErrorException: Unknown column 'type' in 'field list'

@clementchong
Copy link
Contributor Author

Looks like this is the issue. Not sure if we can remove it.

image

@scottsut
Copy link
Contributor

scottsut commented Jul 1, 2022

Looks like this is the issue. Not sure if we can remove it.

image

It is not recommended to use the code of dev branch in production !!

Database changes in the code of dev branch are not added to the migration script, which is the problem you mentioned above.

@clementchong
Copy link
Contributor Author

ok, I would ignore this column change. Thank you!

@clementchong
Copy link
Contributor Author

Quick question: Code is merged, would you prefer I always close the issues here on to leave them open? Not sure if you are tracking anything outside and then closing on your end.

@scottsut
Copy link
Contributor

scottsut commented Jul 1, 2022

Never mind, you can always close issue after code is merged, other topics can be discussed in the new issue.

@clementchong
Copy link
Contributor Author

ok thank you!

@scottsut scottsut added status: volunteer wanted Need more input from community. and removed volunteer wanted labels Mar 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement New feature or request status: volunteer wanted Need more input from community.
Projects
None yet
Development

No branches or pull requests

3 participants