Skip to content

Inline TransportReplAct#createReplicatedOperation #41197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 16, 2019
Merged

Inline TransportReplAct#createReplicatedOperation #41197

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
667cfd8
Inline TransportReplAct#createReplicatedOperation
DaveCTurner Apr 15, 2019
fb1f7eb
fail in try block
DaveCTurner Apr 15, 2019
1350c0f
No need to wrap the listener
DaveCTurner Apr 15, 2019
a26a986
Less usage of primaryShardReference
DaveCTurner Apr 15, 2019
c5dc9f9
Revert "Less usage of primaryShardReference"
DaveCTurner Apr 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion server/src/main/java/org/elasticsearch/action/support/replication/ReplicationOperation.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ public class ReplicationOperation<
private final long primaryTerm;

// exposed for tests
final ActionListener<PrimaryResultT> resultListener;
private final ActionListener<PrimaryResultT> resultListener;

private volatile PrimaryResultT primaryResult = null;

Expand Down
64 changes: 29 additions & 35 deletions ...rc/main/java/org/elasticsearch/action/support/replication/TransportReplicationAction.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -357,37 +357,37 @@ public void handleException(TransportException exp) {
});
} else {
setPhase(replicationTask, "primary");
createReplicatedOperation(primaryRequest.getRequest(),
ActionListener.wrap(result -> result.respond(
new ActionListener<>() {
@Override
public void onResponse(Response response) {
if (syncGlobalCheckpointAfterOperation) {
final IndexShard shard = primaryShardReference.indexShard;
try {
shard.maybeSyncGlobalCheckpoint("post-operation");
} catch (final Exception e) {
// only log non-closed exceptions
if (ExceptionsHelper.unwrap(
e, AlreadyClosedException.class, IndexShardClosedException.class) == null) {
// intentionally swallow, a missed global checkpoint sync should not fail this operation
logger.info(
new ParameterizedMessage(
"{} failed to execute post-operation global checkpoint sync", shard.shardId()), e);
}
}
}
primaryShardReference.close(); // release shard operation lock before responding to caller
setPhase(replicationTask, "finished");
onCompletionListener.onResponse(response);
}

@Override
public void onFailure(Exception e) {
handleException(primaryShardReference, e);
final ActionListener<Response> referenceClosingListener = ActionListener.wrap(response -> {
Copy link
Contributor

@henningandersen henningandersen Apr 15, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find the separation into two listeners artificial and a bit confusing. I suggest something like following instead:

                    final ActionListener<Response> globalCheckpointSyncingListener = ActionListener.wrap(response -> {
                        if (syncGlobalCheckpointAfterOperation) {
                            final IndexShard shard = primaryShardReference.indexShard;
                            try {
                                shard.maybeSyncGlobalCheckpoint("post-operation");
                            } catch (final Exception e) {
                                // only log non-closed exceptions
                                if (ExceptionsHelper.unwrap(
                                    e, AlreadyClosedException.class, IndexShardClosedException.class) == null) {
                                    // intentionally swallow, a missed global checkpoint sync should not fail this operation
                                    logger.info(
                                        new ParameterizedMessage(
                                            "{} failed to execute post-operation global checkpoint sync", shard.shardId()), e);
                                }
                            }
                        }
                        primaryShardReference.close(); // release shard operation lock before responding to caller
                        setPhase(replicationTask, "finished");
                        onCompletionListener.onResponse(response);
                    }, e -> handleException(primaryShardReference, e));

                    new ReplicationOperation<>(primaryRequest.getRequest(), primaryShardReference,
                        ActionListener.wrap(result -> result.respond(globalCheckpointSyncingListener),
                            globalCheckpointSyncingListener::onFailure),
                        newReplicasProxy(), logger, actionName, primaryRequest.getPrimaryTerm()).execute();

Copy link
Contributor Author

@DaveCTurner DaveCTurner Apr 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In isolation I agree, but this separation will be important in a followup so I hope it's ok to leave it like it is. The global checkpoint syncing is the responsibility of the primary, whereas the cleanup of the replication task and the primaryShardReference is the responsibility of the reroute/delegation phase.

Copy link
Contributor

@henningandersen henningandersen Apr 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

primaryShardReference.close(); // release shard operation lock before responding to caller
setPhase(replicationTask, "finished");
onCompletionListener.onResponse(response);
}, e -> handleException(primaryShardReference, e));

final IndexShard shard = primaryShardReference.indexShard;

final ActionListener<Response> globalCheckpointSyncingListener = ActionListener.wrap(response -> {
if (syncGlobalCheckpointAfterOperation) {
try {
shard.maybeSyncGlobalCheckpoint("post-operation");
} catch (final Exception e) {
// only log non-closed exceptions
if (ExceptionsHelper.unwrap(
e, AlreadyClosedException.class, IndexShardClosedException.class) == null) {
// intentionally swallow, a missed global checkpoint sync should not fail this operation
logger.info(
new ParameterizedMessage(
"{} failed to execute post-operation global checkpoint sync",
primaryShardReference.routingEntry().shardId()), e);
Copy link
Contributor

@henningandersen henningandersen Apr 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow this change, I cannot figure out how this makes a difference. I think using just shard.shardId() is simpler unless there is a reason for this?

Copy link
Contributor Author

@DaveCTurner DaveCTurner Apr 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More foreshadowing of changes to come, but I can defer this until later.

}
}), e -> handleException(primaryShardReference, e)
), primaryShardReference).execute();
}
}
referenceClosingListener.onResponse(response);
}, referenceClosingListener::onFailure);

new ReplicationOperation<>(primaryRequest.getRequest(), primaryShardReference,
ActionListener.wrap(result -> result.respond(globalCheckpointSyncingListener), referenceClosingListener::onFailure),
newReplicasProxy(), logger, actionName, primaryRequest.getPrimaryTerm()).execute();
}
} catch (Exception e) {
handleException(primaryShardReference, e);
Expand All @@ -405,12 +405,6 @@ public void onFailure(Exception e) {
onCompletionListener.onFailure(e);
}

protected ReplicationOperation<Request, ReplicaRequest, PrimaryResult<ReplicaRequest, Response>> createReplicatedOperation(
Request request, ActionListener<PrimaryResult<ReplicaRequest, Response>> listener,
PrimaryShardReference primaryShardReference) {
return new ReplicationOperation<>(request, primaryShardReference, listener,
newReplicasProxy(), logger, actionName, primaryRequest.getPrimaryTerm());
}
}

public static class PrimaryResult<ReplicaRequest extends ReplicationRequest<ReplicaRequest>,
Expand Down
Loading