Re: Fw: ZSH history file VS. UTF-8 data

[lilydjwg <lilydjwg@xxxxxxxxx>]

On Wed, Mar 02, 2011 at 09:43:36AM +0000, Peter Stephenson wrote:
> I got this.  I haven't investigated.  Replies will need to be copied
> directly.
> 
> 
> Begin forwarded message:
> 
> Date: Wed, 2 Mar 2011 11:49:29 +0900
> From: ıɥɔıɐʇ ɐʇɐqɐʍɐʞ <kawabata.taichi@xxxxxxxxx>
> To: <pws@xxxxxxx>
> Subject: ZSH history file VS. UTF-8 data
> 
> 
> Dear sir,
> 
> I'm sorry to disturb you.
> 
> I've been a user of ZSH  for a long time and I really
> appreciate for the developer of this great software.
> 
> Recently, I've got one trouble in ZSH history file, and as I tried to
> solve it by
> surveying Webs and Sources, with no help.  As of it, knowing that this
> disturbs you,
> I'm trying to ask for any help.
> 
> The trouble is as follows:
> 
> When I try to use UTF-8 file name in shell command, ZHS history file
> seems to save
> it with "meta code".
> 
> For example, executing
>  $ ls \346\226\207\345\255\227  (octal expression of "ls 文字”）
> results in histfile
>  $ ls \346\203\266\203\247\2\345\255\203\267
> 
> That is, when 0x80-0x9F characters are used, then always 0x83 Meta
> character is inserted and following character is bit shifted, resulting
> garbage history.
> 
> Any way to avoid this situation?
> 
> Any help is really appreciated,
> 

I met the same problem, and had a look at the zsh source code, attached
is the C program I use to deal with this. However, I have not found an
option to turn this feature off or why the history file is encoded in
this way.

-- 
Best regards,
lilydjwg

Linux Vim Python 我的博客
http://lilydjwg.is-programmer.com/

//=====================================================================
// 让 zsh 的历史记录可读
//=====================================================================
#include<unistd.h>
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
//---------------------------------------------------------------------
void readhist();
void writehist();
void usage();
static int ismeta(unsigned char ch);
//---------------------------------------------------------------------
int main(int argc, char** argv){
  if(argc == 1)
    readhist();
  else if(argc == 2){
    if(!strcmp(argv[1], "read"))
      readhist();
    else if(!strcmp(argv[1], "write"))
      writehist();
    else
      usage();
  }else
    usage();
  return 0;
}
void readhist(){
  unsigned char c, d;
  int change = 0;
  while(read(0, &c, 1)){
    if(c != 0x83){
      if(change){
	d = c ^ 32;
      }else
	d = c;
      write(1, &d, 1);
      change = 0;
    }else
      change = 1;
  }
}
void writehist(){
  unsigned char c, d;
  while(read(0, &c, 1)){
    if(ismeta(c)){
      d = 0x83;
      write(1, &d, 1);
      d = c ^ 32;
      write(1, &d, 1);
    }else
      write(1, &c, 1);
  }
}
void usage(){
  fprintf(stderr, "Usage: zhist [read|write]\n"
      "\tzhist reads from stdin and outputs to stdout, "
      "with the content changes\nto be read by human being "
      "or by the zsh program.\n");
  exit(1);
}
static int ismeta(unsigned char ch){
  return (ch > 0x83 && ch < 0x9e)
    || ch == 0xa0 || ch == 0x83 || ch == 0;
}
//=====================================================================