Using the history expansion modifier 'a' results in excessive lstat calls

X-seq: zsh-workers 50259

From: Stefan Radziuk <sradziuk@xxxxxxxxxxxxxxx>

To: "zsh-workers@xxxxxxx" <zsh-workers@xxxxxxx>

Subject: Using the history expansion modifier 'a' results in excessive lstat calls

Date: Fri, 20 May 2022 10:08:01 +0000

Accept-language: en-US

Arc-authentication-results: i=2; mx.microsoft.com 1; spf=fail (sender ip is 199.20.13.32) smtp.rcpttodomain=zsh.org smtp.mailfrom=pdtpartners.com; dmarc=fail (p=none sp=none pct=100) action=none header.from=pdtpartners.com; dkim=fail (body hash did not verify) header.d=pdtpartners.com; arc=fail (47)

Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=pdtpartners.com; dmarc=pass action=none header.from=pdtpartners.com; dkim=pass header.d=pdtpartners.com; arc=none

Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=no7odjjTpUEYEibVaw6lUEjE4gNcZXmxYaVyCHx1DW4=; b=D56TAj/71BA4nUHntVhsVoo9KGpyOmvCJ8pRDbJE5/FYgInfWaqaM44FUcUrJv2rvcQDbFj2xsbApYHKv105DcvTYQvx+NpcXPDZnY/uDuDAZq56uynVhTITKTXvzg/fRdcwFalIAWyteA2QVL5rH6mnM0yZH05VUT/8tHds1VkMbQYOrpPyBl8hGiCNx+C4cQioWSjcJ0KG/ect7bJbFUEHVE5RrKKeGStFfl8Nz6vAOSR0vWsdX4OOab483n2CSVV7dvJd+JsL9GXlRGyu/FQIVDVnyzRVe+N4Q6UKz1FZDhJct3wsCSRZmgPDgYroCyxNoV4EKjqhvTXtCeRn0A==

Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=79C6dmkFZb/CGCKFPZqpNhot8GhnDM03J9e6lbWPpDI=; b=dBcvLyjQgs3JXJYvAbLiHkYGdJuovJva8E2WaySyl8+GBcY5vYoJ+BYU8ATs1XGGigFvicpBn5gDrfVdwakWBffP63EQl2v8gIjrlQeXXM/FSIqj0gELIOPw9DMejzyinqGeLScTzFIQWdDYFblrTjWGQ3Nq062BjI4LrVywIMMAUH5TY8LYRC3EXcgU8kvk9CRCZUYhFGnkiGhm0tc2CXqKEEQy8Vc6C4LV+5+u+430CAZbDvAGN1V3dvbMQPluzSIg6Ilw4Z6+5d/M1WNfcWNI/z8qDfaJtEiP2GwxoDbXvfnTn77ln2QfwQUJW/bHflYGLHF4BxuR76GJZCazHQ==

Arc-seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=fail; b=kSbrRvdGXFP2zJQj9KfF6v0NyMn0/7qipKnlgvJfxgEZ/Nn7CQfB4LjpV68g/OwI5K3eN1U5/siPhdN3qhdtGRzLuHXKV/3M+pGl0Z86p0ASBx1+Odb8ciz+PosuwgxDDNnk9xoXhCEg5GbgbdIEjL6IKaGKjaFaUxLFERqxzxL6Zmzo2oQBYmgOy+9wUuo5xA4m7NHUfjHhdVH4qgK9E/0pwKYbQ4vM8hKcpevHvE4EqtJ04fdUysZbFuJRbxXulPP5zUW6COXeCy7Vov7ONsNEu2Ct+/P/8xSWp8cjSIcBq3EgDoj1hiVUGVzcBKnHw0dovRbexZZ2ebterK+Y4A==

Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jWTKwmZqIgf8jJCqG6dgzJp3G5rdpzNWha56NX7jQhGLy9QAtHS7oCLhgaa8oFdrjr57KsCYig++qHai32fHQd2v6rnPzdMfRDR2uNTmziLFLzB6NUWrhyBtxefJPURqueZJSwo8i3eTMLEshlPPeqXZAAlYYd6tG/frL8sEnlrcyWpr0s7lw/HFK6F/SPMQgIyc5mCeACM6nTWlPrpUE0y0fNkjHZih8u8ywsT8F/dkXI5yxEwQkm338F61KQOqx3HvFIUr087HmYL1eIgb/F688HuDrbPJoSQiVusChtD+XcLrLjm0YMo+P6Sh/Gt/G5uTIc8WUvQVOO6q9cIUOg==

Archived-at: <https://zsh.org/workers/50259>

List-id: <zsh-workers.zsh.org>

Thread-index: AQHYbDEPYF0J/kRF40qfVUNFCmBzxg==

Thread-topic: Using the history expansion modifier 'a' results in excessive lstat calls

zsh uses zgetdir to evaluate ${FOO:a}. When in a directory in a different filesystem to its parent, zgetdir will lstat many entries in the parent. This leads to slowness, especially when some of the lstatted entries are on remote filesystems.

This behaviour can be avoided by making zsh use getcwd instead (build zsh with USE_GETCWD). Interestingly, getcwd will also be used in the regular zsh build if zgetdir fails (see the implementation of zgetcwd).

I have looked through some threads in the mailing list to find out why zgetdir is being used over getcwd in the first place. It seems it was implemented this way in the 1990s to work around a bug in some implementations of getcwd, which should not be a concern on modern systems.

I was wondering if zgetdir is still the right way to do this? Are there use cases where it is preferred over getcwd? Maybe zgetcwd could simply use getcwd primarily (i.e. not as fallback), or perhaps USE_GETCWD could be enabled by default (currently it is only enabled on QNX builds).

This communication is intended only for the addressee(s), may contain confidential, privileged or proprietary information, and may be protected by US and other laws. Your acceptance of this communication constitutes your agreement to keep confidential all the confidential information contained in this communication, as well as any information derived by you from the confidential information contained in this communication. We do not waive any confidentiality by misdelivery.

If you receive this communication in error, any use, dissemination, printing or copying is strictly prohibited; please destroy all electronic and paper copies and notify the sender immediately. Nothing in this email is intended to constitute (1) investment or trading advice or recommendations or any advertisement or (2) a solicitation of an investment in any jurisdiction in which such a solicitation would be unlawful.

Please note that PDT Partners UK, LLP, including its affiliates, reserves the right to intercept, archive, monitor and review all communications to and from its network.

Reverse Date

Date

Thread

Author