[ph-perl] Re: [plug] perl - removing duplicate line entries …

Top Page

Reply to this message
Author: ph-perl Mailing List
To: roderick tapang, The Main Philippine Linux Users' Group (PLUG) Discussion List
CC: ph-perl
Subject: [ph-perl] Re: [plug] perl - removing duplicate line entries in a file
On Wed, May 04, 2005 at 10:12:18AM +0800, roderick tapang wrote:
> how do you remove duplicate line entries in a file
> using perl? we have some logfiles that are xferred
> to a windows machine (don't ask why) for analysis
> and we only have ActivePerl available in it and one
> staff needs to have the duplicate lines remove.
> i really don't do perl so can anyone give out some
> lines of codes for this?
> (i posted this in ph-perl but didn't get any response
> yet).

Try this:

# usage: remove_dup_in_place.pl file1 file2 ... fileN

while (<>) {
  if ($ARGV ne $oldargv) {
    open(ARGVOUT, ">$ARGV");
    $oldargv = $ARGV;
    undef %hash;
  print ARGVOUT sort keys %hash 
    if (eof and defined fileno(ARGVOUT));


$_=q:; # SHERWIN #
?=pack q$C$,hex$1;
Philippine Linux Users' Group (PLUG) Mailing List
plug@??? (#PLUG @ irc.free.net.ph)
Read the Guidelines: http://linux.org.ph/lists
Searchable Archives: http://archives.free.net.ph