Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

解决日志系统未初始化完成时,apollo 的加载日志没法输出问题 #3677

Merged
merged 29 commits into from
May 23, 2021
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
fe75b4d
add tech-support-qq-4.png
klboke May 16, 2019
99bf97a
Update README.md
klboke May 16, 2019
1579f41
Merge remote-tracking branch 'upstream/master'
klboke Nov 11, 2020
9f65eed
Merge remote-tracking branch 'upstream/master'
klboke Dec 9, 2020
d7d3fd9
Enhance the user experience in the scenario of submitting duplicate keys
klboke Dec 11, 2020
7329fab
Merge branch 'master' into master
nobodyiam Dec 12, 2020
5def448
Modify the key-value conflict exception prompt, adjust the code style
klboke Dec 12, 2020
6942564
Merge branch 'master' into master
nobodyiam Dec 12, 2020
0fc1f91
Merge remote-tracking branch 'upstream/master'
klboke Dec 19, 2020
61ad016
Merge remote-tracking branch 'origin/master'
klboke Dec 19, 2020
587ce33
rge remote-tracking branch 'upstream/master'
klboke Mar 1, 2021
9de6563
Merge remote-tracking branch 'upstream/master'
klboke Mar 8, 2021
9aab632
Merge remote-tracking branch 'upstream/master'
klboke Mar 25, 2021
050dd0a
Merge branch 'ctripcorp:master' into master
klboke May 10, 2021
5a64035
Merge branch 'master' of github.com:klboke/apollo
klboke May 10, 2021
d053f4a
feat[apollo-client]: Solve the problem that the log before the log sy…
klboke May 11, 2021
41acba5
Merge branch 'master' into DeferredLog
nobodyiam May 15, 2021
b1ce641
refactor(DeferredLog): Delay log refactoring
klboke May 19, 2021
6357c8f
Merge remote-tracking branch 'origin/DeferredLog' into DeferredLog
klboke May 19, 2021
93560e8
fix(DeferredLog): Fix log level definition error
klboke May 19, 2021
072c58a
feat(DeferredLog): Complete test cases
klboke May 20, 2021
ea37efc
fix(DeferredLog): Fix problems with test cases
klboke May 21, 2021
65c7714
refactor(DeferredLog): Remove useless packages
klboke May 21, 2021
f47fc44
refactor(DeferredLog): DeferredLogCache only stores logs
klboke May 21, 2021
2c4fde5
refactor(DeferredLog): AbstractConfigFile adds delay log
klboke May 21, 2021
2532a7f
refactor(DeferredLog): Improve delayed log implementation
klboke May 21, 2021
6ea77e5
Merge branch 'master' into DeferredLog
nobodyiam May 21, 2021
ee2339c
fix(DeferredLog): Fix the problem of test case compilation failure, a…
klboke May 21, 2021
4eecb14
refactor(DeferredLog): DeferredLogCache method reconstruction
klboke May 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package com.ctrip.framework.apollo.internals;

import com.ctrip.framework.apollo.core.utils.DeferredLogUtil;
import com.ctrip.framework.apollo.enums.ConfigSourceType;
import java.io.IOException;
import java.io.InputStream;
Expand Down Expand Up @@ -58,8 +59,9 @@ private void initialize() {
updateConfig(m_configRepository.getConfig(), m_configRepository.getSourceType());
} catch (Throwable ex) {
Tracer.logError(ex);
logger.warn("Init Apollo Local Config failed - namespace: {}, reason: {}.",
m_namespace, ExceptionUtil.getDetailMessage(ex));
String logWarnMsg = "Init Apollo Local Config failed - namespace: {}, reason: {}.";
logger.warn(logWarnMsg, m_namespace, ExceptionUtil.getDetailMessage(ex));
DeferredLogUtil.warn(logger, logWarnMsg, m_namespace, ExceptionUtil.getDetailMessage(ex));
klboke marked this conversation as resolved.
Show resolved Hide resolved
} finally {
//register the change listener no matter config repository is working or not
//so that whenever config repository is recovered, config could get changed
Expand Down Expand Up @@ -92,7 +94,9 @@ public String getProperty(String key, String defaultValue) {
}

if (value == null && m_configProperties.get() == null && m_warnLogRateLimiter.tryAcquire()) {
logger.warn("Could not load config for namespace {} from Apollo, please check whether the configs are released in Apollo! Return default value now!", m_namespace);
String logWarnMsg = "Could not load config for namespace {} from Apollo, please check whether the configs are released in Apollo! Return default value now!";
logger.warn(logWarnMsg, m_namespace);
DeferredLogUtil.warn(logger, logWarnMsg);
}

return value == null ? defaultValue : value;
Expand Down Expand Up @@ -219,7 +223,9 @@ private Properties loadFromResource(String namespace) {
properties.load(in);
} catch (IOException ex) {
Tracer.logError(ex);
logger.error("Load resource config for namespace {} failed", namespace, ex);
String errMsg = "Load resource config for namespace {} failed";
logger.error(errMsg, namespace, ex);
DeferredLogUtil.error(logger, errMsg, ex);
} finally {
try {
in.close();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import com.ctrip.framework.apollo.core.ConfigConsts;
import com.ctrip.framework.apollo.core.enums.Env;
import com.ctrip.framework.apollo.core.spi.MetaServerProvider;
import com.ctrip.framework.apollo.core.utils.DeferredLogUtil;
import com.ctrip.framework.foundation.Foundation;
import com.google.common.base.Strings;
import org.slf4j.Logger;
Expand Down Expand Up @@ -36,10 +37,14 @@ private String initMetaServerAddress() {
}

if (Strings.isNullOrEmpty(metaAddress)) {
logger.warn("Could not find meta server address, because it is not available in neither (1) JVM system property 'apollo.meta', (2) OS env variable 'APOLLO_META' (3) property 'apollo.meta' from server.properties nor (4) property 'apollo.meta' from app.properties");
String logWarnMsg = "Could not find meta server address, because it is not available in neither (1) JVM system property 'apollo.meta', (2) OS env variable 'APOLLO_META' (3) property 'apollo.meta' from server.properties nor (4) property 'apollo.meta' from app.properties";
logger.warn(logWarnMsg);
DeferredLogUtil.warn(logger, logWarnMsg);
} else {
metaAddress = metaAddress.trim();
logger.info("Located meta services from apollo.meta configuration: {}!", metaAddress);
String logMsg = "Located meta services from apollo.meta configuration: {}!";
logger.info(logMsg, metaAddress);
DeferredLogUtil.info(logger, logMsg, metaAddress);
}

return metaAddress;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import com.ctrip.framework.apollo.Config;
import com.ctrip.framework.apollo.ConfigService;
import com.ctrip.framework.apollo.core.ConfigConsts;
import com.ctrip.framework.apollo.core.utils.DeferredLogUtil;
import com.ctrip.framework.apollo.spring.config.ConfigPropertySourceFactory;
import com.ctrip.framework.apollo.spring.config.PropertySourcesConstants;
import com.ctrip.framework.apollo.spring.util.SpringInjector;
Expand Down Expand Up @@ -91,7 +92,8 @@ public void initialize(ConfigurableApplicationContext context) {
protected void initialize(ConfigurableEnvironment environment) {

if (environment.getPropertySources().contains(PropertySourcesConstants.APOLLO_BOOTSTRAP_PROPERTY_SOURCE_NAME)) {
//already initialized
//already initialized , replay the logs that were printed before the logging system was initialized
klboke marked this conversation as resolved.
Show resolved Hide resolved
DeferredLogUtil.replayTo();
klboke marked this conversation as resolved.
Show resolved Hide resolved
return;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;

import com.ctrip.framework.apollo.core.utils.DeferredLogUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down Expand Up @@ -90,18 +91,19 @@ private static void initMetaServerAddress(Env env) {
for (MetaServerProvider provider : metaServerProviders) {
metaAddress = provider.getMetaServerAddress(env);
if (!Strings.isNullOrEmpty(metaAddress)) {
logger.info("Located meta server address {} for env {} from {}", metaAddress, env,
provider.getClass().getName());
String logMsg = "Located meta server address {} for env {} from {}";
logger.info(logMsg, metaAddress, env, provider.getClass().getName());
DeferredLogUtil.info(logger, logMsg, metaAddress, env, provider.getClass().getName());
break;
}
}

if (Strings.isNullOrEmpty(metaAddress)) {
// Fallback to default meta address
metaAddress = DEFAULT_META_URL;
logger.warn(
"Meta server address fallback to {} for env {}, because it is not available in all MetaServerProviders",
metaAddress, env);
String logWarnMsg = "Meta server address fallback to {} for env {}, because it is not available in all MetaServerProviders";
logger.warn(logWarnMsg, metaAddress, env);
DeferredLogUtil.warn(logger, logWarnMsg, metaAddress, env);
}

metaServerAddressCache.put(env, metaAddress.trim());
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
package com.ctrip.framework.apollo.core.utils;

import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import org.slf4j.Logger;
import org.slf4j.event.Level;

import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;

/**
* Delayed log printing utility class, used only for logging when Apollo is initialized
* @author kl (http://kailing.pub)
* @since 2021/5/11
*/
public final class DeferredLogUtil {

public static final int MAX_LOG_SIZE = 1000;
private static final AtomicInteger LOG_INDEX = new AtomicInteger(0);
private static final Cache<Integer, Line> LOG_CACHE = CacheBuilder.newBuilder()
.maximumSize(MAX_LOG_SIZE)
.expireAfterWrite(1, TimeUnit.MINUTES)
.build();

private DeferredLogUtil(){}

public static void debug(Logger logger, String message, Object... objects) {
add(logger, Level.DEBUG, message, objects, null);
}

public static void info(Logger logger, String message, Object... objects) {
add(logger, Level.INFO, message, objects, null);
}

public static void warn(Logger logger, String message, Object... objects) {
add(logger, Level.WARN, message, objects, null);
}

public static void error(Logger logger, String message, Throwable throwable) {
add(logger, Level.WARN, message, null, throwable);
}

private static void add(Logger logger, Level level, String message, Object[] objects, Throwable throwable) {
Line logLine = new Line(level, message, objects, throwable, logger);
LOG_CACHE.put(LOG_INDEX.incrementAndGet(), logLine);
}

public static void replayTo() {
for (int i = 1; i <= LOG_INDEX.get(); i++) {
Line logLine = LOG_CACHE.getIfPresent(i);
assert logLine != null;
Logger logger = logLine.getLogger();
Level level = logLine.getLevel();
String message = logLine.getMessage();
Object[] objects = logLine.getObjects();
Throwable throwable = logLine.getThrowable();
logTo(logger, level, message, objects, throwable);
}
clear();
}

private static void clear(){
LOG_CACHE.invalidateAll();
LOG_INDEX.set(0);
}

public static long logSize(){
return LOG_CACHE.size();
}

static void logTo(Logger logger, Level level, String message, Object[] objects, Throwable throwable) {
switch (level) {
case DEBUG:
logger.debug(message, objects);
return;
case INFO:
logger.info(message, objects);
return;
case WARN:
logger.warn(message, objects);
return;
case ERROR:
logger.error(message, throwable);
break;
default:
throw new IllegalStateException("Unexpected value: " + level);
}
}

static class Line {

private final Level level;

private final String message;

private final Object[] objects;

private final Throwable throwable;

private final Logger logger;

Line(Level level, String message, Object[] objects, Throwable throwable, Logger logger) {
this.level = level;
this.message = message;
this.objects = objects;
this.throwable = throwable;
this.logger = logger;
}

public Object[] getObjects() {
return objects;
}

public Logger getLogger() {
return logger;
}

Level getLevel() {
return this.level;
}

String getMessage() {
return this.message;
}

Throwable getThrowable() {
return this.throwable;
}

}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import java.nio.charset.StandardCharsets;
import java.util.Properties;

import com.ctrip.framework.apollo.core.utils.DeferredLogUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down Expand Up @@ -33,7 +34,9 @@ public void initialize() {

initialize(in);
} catch (Throwable ex) {
logger.error("Initialize DefaultApplicationProvider failed.", ex);
String errMsg = "Initialize DefaultApplicationProvider failed.";
logger.error(errMsg, ex);
DeferredLogUtil.error(logger, errMsg, ex);
}
}

Expand All @@ -52,7 +55,9 @@ public void initialize(InputStream in) {
initAppId();
initAccessKey();
} catch (Throwable ex) {
logger.error("Initialize DefaultApplicationProvider failed.", ex);
String errMsg = "Initialize DefaultApplicationProvider failed.";
logger.error(errMsg, ex);
DeferredLogUtil.error(logger, errMsg, ex);
}
}

Expand Down Expand Up @@ -97,57 +102,66 @@ private void initAppId() {
m_appId = System.getProperty("app.id");
if (!Utils.isBlank(m_appId)) {
m_appId = m_appId.trim();
logger.info("App ID is set to {} by app.id property from System Property", m_appId);
String logMsg = "App ID is set to {} by app.id property from System Property";
logger.info(logMsg, m_appId);
DeferredLogUtil.info(logger, logMsg, m_appId);
return;
}

//2. Try to get app id from OS environment variable
m_appId = System.getenv("APP_ID");
if (!Utils.isBlank(m_appId)) {
m_appId = m_appId.trim();
logger.info("App ID is set to {} by APP_ID property from OS environment variable", m_appId);
String logMsg = "App ID is set to {} by APP_ID property from OS environment variable";
logger.info(logMsg, m_appId);
DeferredLogUtil.info(logger, logMsg, m_appId);
return;
}

// 3. Try to get app id from app.properties.
m_appId = m_appProperties.getProperty("app.id");
if (!Utils.isBlank(m_appId)) {
m_appId = m_appId.trim();
logger.info("App ID is set to {} by app.id property from {}", m_appId,
APP_PROPERTIES_CLASSPATH);
String logMsg = "App ID is set to {} by app.id property from {}";
logger.info(logMsg, m_appId, APP_PROPERTIES_CLASSPATH);
DeferredLogUtil.info(logger, logMsg, m_appId, APP_PROPERTIES_CLASSPATH);
return;
}

m_appId = null;
logger.warn("app.id is not available from System Property and {}. It is set to null",
APP_PROPERTIES_CLASSPATH);
String logWarnMsg = "app.id is not available from System Property and {}. It is set to null";
logger.warn(logWarnMsg, APP_PROPERTIES_CLASSPATH);
DeferredLogUtil.warn(logger, logWarnMsg, APP_PROPERTIES_CLASSPATH);
}

private void initAccessKey() {
// 1. Get accesskey secret from System Property
accessKeySecret = System.getProperty("apollo.accesskey.secret");
if (!Utils.isBlank(accessKeySecret)) {
accessKeySecret = accessKeySecret.trim();
logger
.info("ACCESSKEY SECRET is set by apollo.accesskey.secret property from System Property");
String logMsg = "ACCESSKEY SECRET is set by apollo.accesskey.secret property from System Property";
logger.info(logMsg);
DeferredLogUtil.info(logger, logMsg);
return;
}

//2. Try to get accesskey secret from OS environment variable
accessKeySecret = System.getenv("APOLLO_ACCESSKEY_SECRET");
if (!Utils.isBlank(accessKeySecret)) {
accessKeySecret = accessKeySecret.trim();
logger.info(
"ACCESSKEY SECRET is set by APOLLO_ACCESSKEY_SECRET property from OS environment variable");
String logMsg = "ACCESSKEY SECRET is set by APOLLO_ACCESSKEY_SECRET property from OS environment variable";
logger.info(logMsg);
DeferredLogUtil.info(logger, logMsg);
return;
}

// 3. Try to get accesskey secret from app.properties.
accessKeySecret = m_appProperties.getProperty("apollo.accesskey.secret");
if (!Utils.isBlank(accessKeySecret)) {
accessKeySecret = accessKeySecret.trim();
logger.info("ACCESSKEY SECRET is set by apollo.accesskey.secret property from {}",
APP_PROPERTIES_CLASSPATH);
String logMsg = "ACCESSKEY SECRET is set by apollo.accesskey.secret property from {}";
logger.info(logMsg, APP_PROPERTIES_CLASSPATH);
DeferredLogUtil.info(logger, logMsg, APP_PROPERTIES_CLASSPATH);
return;
}

Expand Down
Loading