{data dendrites} exploring data science with Python et al.

Use unique( ) to drop duplicates

In a recent post, I reviewed how you can use union() to get unique members out of a set. A faster way to do that is to use unique() function:

Out:

[3 4 8 9 8 2 4 9 2 7 3 3 6 9 4
 5 3 0 0 10 7 6 9 9 2 10 0 5 9 5]
[3,4,8,9,2,7,6]
[5,3,0,10,7,6,9,2]
[3,5,4,8,0,9,10,2,7,6]
[0,2,3,4,5,6,7,8,9,10]

Compare times:

n = 3
@time [ union(a,b) for i in 1:10^n ]
@time [ unique([a,b]) for i in 1:10^n ];

Out:

elapsed time: 0.007959665 seconds (2335432 bytes allocated)
elapsed time: 0.002394242 seconds (1983432 bytes allocated)
comments powered by Disqus