# unique

Definition: | sequence res = unique(sequence s, string options="SORT")
Description: | Returns s with duplicate elements removed. s: the sequence from which to remove duplicates. options: "SORT" (the default) performs a standard sort on s before removing duplicates, "PRESORTED" indicates the initial sort is unnecesary, "STABLE" (==="INPLACE") retains the original order of any unique entries, but uses a slightly slower dictionary-based method of removing duplicates. A fatal error occurs if options is not one of those (or not all upper-case). If "PRESORTED" is specified, but s is not actually sorted, then only adjacent duplicates are removed. Note that under with javascript_semantics `result = unique(result)` should be replaced with `result = unique(deep_copy(result))` when result is not routine-local, and similarly when res and s are different variables, that is unless options is "PRESORTED" or "STABLE"/"INPLACE".
pwa/p2js: | Supported, may however trigger p2js violations, that usually need unique(deep_copy(s)) to fix.
Example: |
    sequence s = { 4,7,9,7,2,5,5,9,0,4,4,5,6,5}
    ?unique(s, "STABLE")            --=> {4,7,9,2,5,0,6}
    ?unique(s, "SORT")              --=> {0,2,4,5,6,7,9}
    ?unique(s, "PRESORTED")         --=> {4,7,9,7,2,5,9,0,4,5,6,5}
    ?unique(sort(s), "PRESORTED")   --=> {0,2,4,5,6,7,9}
Implementation: | See builtins\punique.e (an autoinclude) for details of the actual implementation.
Technicalia | This is functionally equivalent to the Euphoria remove_dups() routine in std\sequence.e except that: * the options in the latter is an integer which should be one of the provided RD_XXX enums, * RD_INPLACE uses find() rather than a dictionary and may therefore be exponentially slower on large data sets, * the Euphoria default is PRESORTED whereas the Phix default is SORT, and * the Euphoria routine is not an autoinclude. I elected to have string options in preference to adding yet more literal (integer) constants to syminit(). There is in fact a remove_dups() routine in builtins\punique.e, along with a global enum RD_INPLACE, RD_PRESORTED, RD_SORT which offers better compatibility with Euphoria, but they can only be used following an explicit `include builtins\punique.e` statement, which may require a `--/**/` prefix, and a "--/*\n include std/sequence.e\n --*/" to be completely compatible. While this offers perfectly reasonable performance, it may be sub-optimal, simply because building a large sequence and then creating a smaller one from it is not always going to be the fastest possible approach. The dictionary based method was noted above as slightly slower simply because it is creating a potentially significant structure that a plain old sort does not, however something similar to that method (see the source code) might in fact be the fastest way to build a unique list (ie, one that does not need passing through this routine). There will be cases where using this routine actually is the fastest way possible, at least, that is, short of some hideously complicated and extremely ugly inline assembly! I would always just use this and not even consider anything else unless I had at least 5000 or even 50,000 entries.