Skip to content

Conversation

@jiasli
Copy link
Member

@jiasli jiasli commented Apr 26, 2021

Refine #17340

For List Concatenation, using Add (+) is slower compared to INPLACE Add (+=) and extend():

Source: https://blog.finxter.com/python-list-concatenation-add-vs-inplace-add-vs-extend/

This PRs unifies the code for per_call_policies and per_retry_policies and only uses extend() for List Concatenation.

@ghost ghost added the Azure.Core label Apr 26, 2021
@jiasli jiasli changed the title Refine code for per_call_policies and per_retry_policies Use extend() for per_call_policies and per_retry_policies Apr 26, 2021
Comment on lines -126 to -127
for policy in per_call_policies:
policies.append(policy)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for is the slowest.

speed

import time

# Compare runtime of three methods
list_sizes = [i * 400000 for i in range(30)]
runtimes_1 = []  # Method 1: + Operator
runtimes_2 = []  # Method 2: += Operator
runtimes_3 = []  # Method 3: extend()
runtimes_4 = []  # Method 3: for
for size in list_sizes:
    to_add = list(range(size))

    # Get time stamps
    time_0 = time.time()
    lst = [1]
    lst = lst + to_add
    time_1 = time.time()
    lst = [1]
    lst += to_add
    time_2 = time.time()
    lst = [1]
    lst.extend(to_add)
    time_3 = time.time()
    lst = [1]
    for i in to_add:
        lst.append(i)
    time_4 = time.time()
    # Calculate runtimes
    runtimes_1.append((size, time_1 - time_0))
    runtimes_2.append((size, time_2 - time_1))
    runtimes_3.append((size, time_3 - time_2))
    runtimes_4.append((size, time_4 - time_3))
# Plot everything
import matplotlib.pyplot as plt
import numpy as np

runtimes_1 = np.array(runtimes_1)
runtimes_2 = np.array(runtimes_2)
runtimes_3 = np.array(runtimes_3)
runtimes_4 = np.array(runtimes_4)
print(runtimes_1)
print(runtimes_2)
print(runtimes_3)
print(runtimes_4)
plt.plot(runtimes_1[:, 0], runtimes_1[:, 1], label='Method 1: +')
plt.plot(runtimes_2[:, 0], runtimes_2[:, 1], label='Method 2: +=')
plt.plot(runtimes_3[:, 0], runtimes_3[:, 1], label='Method 3: extend()')
plt.plot(runtimes_4[:, 0], runtimes_4[:, 1], label='Method 4: for')
plt.xlabel('list size')
plt.ylabel('runtime (seconds)')
plt.legend()
plt.savefig('speed.png', dpi=400)
plt.show()

Code snippet from https://blog.finxter.com/python-list-concatenation-add-vs-inplace-add-vs-extend/

if isinstance(per_call_policies, Iterable):
for policy in per_call_policies:
policies.append(policy)
policies.extend(per_call_policies)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(For your another PR), do we need list(per_call_policies) here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. list.extend(iterable) by definition takes any iterable.

@ghost ghost added the no-recent-activity There has been no recent activity on this issue. label May 27, 2022
@ghost
Copy link

ghost commented May 27, 2022

Hi @jiasli. Thank you for your interest in helping to improve the Azure SDK experience and for your contribution. We've noticed that there hasn't been recent engagement on this pull request. If this is still an active work stream, please let us know by pushing some changes or leaving a comment. Otherwise, we'll close this out in 7 days.

@ghost ghost closed this Jun 3, 2022
@ghost
Copy link

ghost commented Jun 3, 2022

Hi @jiasli. Thank you for your contribution. Since there hasn't been recent engagement, we're going to close this out. Feel free to respond with a comment containing "/reopen" if you'd like to continue working on these changes. Please be sure to use the command to reopen or remove the "no-recent-activity" label; otherwise, this is likely to be closed again with the next cleanup pass.

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Azure.Core no-recent-activity There has been no recent activity on this issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants