Re: special characters in file names issue

[Roman Perepelitsa <roman.perepelitsa@xxxxxxxxx>]

On Fri, Nov 10, 2023 at 12:17 AM Jim <linux.tech.guy@xxxxxxxxx> wrote:
>
> Hi everyone,
>
> Using scripts, looking to cleanup duplicate files even if named differently.
> The issue I ran into is when a file path contains parentheses. '(' or ')'
>
> Example File Name:  Wallpapers/Web_downloads/05 (1).jpg
>
> The following is part of an anonymous function:
>
> local E
> local -a AllFileNames
> local -A FileNameCkSum
> ...
> for E (${(@)AllFileNames}) {
> [[ -v FileNameCkSum[$E] ]] || FileNameCkSum[$E]=${$(shasum -a 1 $E)[1]} }  # line that fails
> ...
>
> AllFileName contains the result of a glob statement.
>
> Error Message:  (anon):<line no>: invalid subscript

Associative arrays in zsh are finicky when it comes to the content of
their keys. The problem you are experiencing can be distilled to this:

    % typeset -A dict
    % key='('
    % [[ -v dict[$key] ]]
    zsh: invalid subscript

There is no simple quoting that you can apply to $key here: (q), (b),
etc. are all wrong. You could perhaps escape a specific list of
characters ('(', '[', '{' but not '$' or '*') although my memory tells
me that some keys cannot be made to work under `[[ -v ...]]` or
`unset` no matter how you try to escape them. I could be wrong though.

I usually apply one of two workarounds: use hash($x) instead of $x as
a key, or replace the associative array with two plain arrays, one for
keys and another for values. The latter results in O(N) lookup though.

Roman.

P.S.

From the description of your problem I would think that you want file
hashes as keys. Something like this:

    # usage: detect-dup-files [file]..
    function detect-dup-files() {
      emulate -L zsh
      (( ARGC )) || return 0
      local -A seen
      local i files fname hash orig
      files=( $(shasum -ba 256 -- "$@") ) || return
      (( 2 * ARGC == $#files )) || return
      for i in {1..$ARGC}; do
        fname=$argv[i]
        hash=${files[2*i-1]#\\}
        if [[ -n ${orig::=$seen[$hash]} ]]; then
          print -r -- "${(q+)fname} is a dup of ${(q+)orig}"
        else
          seen[$hash]=$fname
        fi
      done
    }

This code has an added advantage of forking only once. It also handles
file names with backslashes and linefeeds in them.