Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix urlsplit #13

Merged
merged 73 commits into from
Jun 15, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
734e3d7
fix failing test by importing
malloxpb Jun 11, 2018
e2abab6
import more variables to not fail the test
malloxpb Jun 11, 2018
1ff8b35
test on py3 only for now
malloxpb Jun 11, 2018
31e224c
decode the result to string on Python 3
malloxpb Jun 11, 2018
23c8b3d
compile cython on py3
malloxpb Jun 11, 2018
3d7eb0e
fix failing test by importing
malloxpb Jun 11, 2018
95260b4
import more variables to not fail the test
malloxpb Jun 11, 2018
790bd94
test on py3 only for now
malloxpb Jun 11, 2018
168830f
decode the result to string on Python 3
malloxpb Jun 11, 2018
89a71b3
compile cython on py3
malloxpb Jun 11, 2018
6b76ede
Merge branch 'fix_tests' of https://github.com/nctl144/urlparse4 into…
malloxpb Jun 11, 2018
1699be0
do not change test cases
malloxpb Jun 11, 2018
978eada
decode result based on input types
malloxpb Jun 11, 2018
3aa1b38
compile in py3
malloxpb Jun 11, 2018
af5eced
remove tests that are not related
malloxpb Jun 11, 2018
c351048
DRY code
malloxpb Jun 11, 2018
5c28dd6
compile in py3
malloxpb Jun 11, 2018
0924478
update the performance test
malloxpb Jun 12, 2018
4165c4f
update the performance test
malloxpb Jun 12, 2018
55de674
return based on input types (DRY)
malloxpb Jun 12, 2018
3a034c2
recompile cython on py3
malloxpb Jun 12, 2018
23e10ca
fix encoding urljoin without base
malloxpb Jun 12, 2018
9a7d433
recompile cython
malloxpb Jun 12, 2018
325b518
return username, password based on input
malloxpb Jun 12, 2018
9d7f33b
recompile cython py3
malloxpb Jun 12, 2018
0974a1e
reduce the performance test time
malloxpb Jun 12, 2018
8c97272
update mozilla files from chromium
malloxpb Jun 13, 2018
f958e75
import func to compare scheme
malloxpb Jun 13, 2018
66f4828
import scheme constants
malloxpb Jun 13, 2018
de26520
import the rest of the functions in urlparse
malloxpb Jun 13, 2018
efa4d52
Define isstandard func
malloxpb Jun 13, 2018
b51b05d
rename the file to its true name
malloxpb Jun 13, 2018
9054383
delete the original file
malloxpb Jun 13, 2018
253ff61
parse urls based on their type
malloxpb Jun 13, 2018
757603f
compile cython py3
malloxpb Jun 13, 2018
0a586b0
fix return error
malloxpb Jun 13, 2018
6b55a9e
recompile cython py3
malloxpb Jun 13, 2018
106eb0f
skip deprecated tests
malloxpb Jun 14, 2018
21af6e9
skip other tests for now
malloxpb Jun 14, 2018
15f6d2a
skip the quoter test
malloxpb Jun 14, 2018
4712e73
put the test url back
malloxpb Jun 14, 2018
9f3628f
add py3.5 to test
malloxpb Jun 14, 2018
7144548
import func to compare scheme
malloxpb Jun 13, 2018
caaafef
import scheme constants
malloxpb Jun 13, 2018
93bb8b3
import the rest of the functions in urlparse
malloxpb Jun 13, 2018
f36681f
Define isstandard func
malloxpb Jun 13, 2018
bc4253c
rename the file to its true name
malloxpb Jun 13, 2018
ad7de66
delete the original file
malloxpb Jun 13, 2018
19126e2
parse urls based on their type
malloxpb Jun 13, 2018
0bb23fa
compile cython py3
malloxpb Jun 13, 2018
222a80c
fix return error
malloxpb Jun 13, 2018
dba7798
recompile cython py3
malloxpb Jun 13, 2018
2de8884
recompile cython py3
malloxpb Jun 14, 2018
9cb095b
Merge branch 'fix_split' of https://github.com/nctl144/urlparse4 into…
malloxpb Jun 14, 2018
3effa97
recompile cython, resolve merge conflicts
malloxpb Jun 14, 2018
0bc77f4
mark as todo
malloxpb Jun 14, 2018
8f160ae
return result based on input type
malloxpb Jun 14, 2018
e0e4a97
compile cython
malloxpb Jun 14, 2018
3c5f21c
skip mailto tests for now
malloxpb Jun 14, 2018
3ece971
add note to discuss
malloxpb Jun 14, 2018
507d427
remove deprecation tests
malloxpb Jun 15, 2018
70da959
avoid repetition
malloxpb Jun 15, 2018
455f810
avoid repetition and indentation
malloxpb Jun 15, 2018
37f39c4
shorten the code
malloxpb Jun 15, 2018
61de690
recompile cython py3
malloxpb Jun 15, 2018
f9bbeb3
rename variables
malloxpb Jun 15, 2018
eaec0a9
move code inside main func, move encode outside timer
malloxpb Jun 15, 2018
9f1a6be
fallback to stdlib when failed to parse
malloxpb Jun 15, 2018
4b7c716
recompile cython
malloxpb Jun 15, 2018
89378bf
use argparse instead
malloxpb Jun 15, 2018
7f57de6
indentation
malloxpb Jun 15, 2018
5d66f3f
std split url based on input type
malloxpb Jun 15, 2018
409129f
recompile Cython
malloxpb Jun 15, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,6 @@ branches:
- /^\d\.\d+\.\d+(rc\d+|\.dev\d+)?$/
matrix:
include:
- python: 2.7
env: TOXENV=py27
- python: 2.7
env: TOXENV=pypy
- python: 2.7
env: TOXENV=pypy3
- python: 3.4
env: TOXENV=py34
- python: 3.5
env: TOXENV=py35
- python: 3.6
Expand Down
52 changes: 44 additions & 8 deletions benchmarks/performance_test.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,53 @@
from urlparse4 import urlsplit, urljoin
from timeit import default_timer as timer

total = 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this file is changes so heavily in this PR, maybe also move all code into some function, e.g. main, and call it at the end? There are two reasons:

  • with a function, it's more natural to add helper functions when the code grows, and we avoid accidentally using global variables
  • PyPy has much better optimizations for local function variables than for globals, so it would be more fair to measure performance inside a function

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense! I will move all the code inside a main() func

import argparse

with open('urls/chromiumUrls.txt') as f:
for url in f:

start = timer()
def main():
parser = argparse.ArgumentParser(description='Measure the time of urlsplit and urljoin')
parser.add_argument('--encode', action='store_true',
help='encode the urls (default: False)')
args = parser.parse_args()

a = urlsplit(url)
encode = args.encode

end = timer()
urlsplit_time = 0

total += end - start
for i in range(5):
with open('urls/chromiumUrls.txt') as f:
for url in f:
if encode:
url = url.encode()

print("the total time is", total, "seconds")
start = timer()
a = urlsplit(url)
end = timer()

urlsplit_time += end - start

print("the urlsplit time with encode in python is", urlsplit_time / 5, "seconds")


urljoin_time = 0

for i in range(5):
with open('urls/chromiumUrls.txt') as f:
for url in f:
partial_url = "/asd"

if encode:
url = url.encode()
partial_url = partial_url.encode()

start = timer()
a = urljoin(url, partial_url)
end = timer()

urljoin_time += end - start

print("the urljoin time with encode in python is", urljoin_time / 5, "seconds")


if __name__ == "__main__":
main()
Loading