Re: [plug] perl - removing duplicate line entries in a file

Top Page

Reply to this message
Author: Peter Santiago
Date:  
To: roderick tapang, The Main Philippine Linux Users' Group (PLUG) Discussion List
Subject: Re: [plug] perl - removing duplicate line entries in a file
roderick tapang wrote:
> how do you remove duplicate line entries in a file
> using perl? we have some logfiles that are xferred
> to a windows machine (don't ask why) for analysis
> and we only have ActivePerl available in it and one
> staff needs to have the duplicate lines remove.
> i really don't do perl so can anyone give out some
> lines of codes for this?
>
> (i posted this in ph-perl but didn't get any response
> yet).
>
> tyvm.
>
> erik


Syntax: dedupes.pl [input] [output]

#!/usr/bin/perl -w
undef %saw;
open (I, $ARGV[0]) or die "\n Couldn't open input file";
$no = 0;
while ($in[$no]=<I>) {$no++};
close I;
open (J, "> $ARGV[1]") or die "\n Couldn't open output file";
@saw{@in}=();
@out=keys %saw;
print J @out;
close J;


Note:  I'm not really a Perl programmer, I just googled for some partial 
codes then combined them into the above.  Tell me if it works correctly 
or not.
-- 
Peter Santiago         peters@???
My website:            www.psinergybbs.com
My spamtrap address:   r34987y@???
_________________________________________________
Philippine Linux Users' Group (PLUG) Mailing List
plug@??? (#PLUG @ irc.free.net.ph)
Read the Guidelines: http://linux.org.ph/lists
Searchable Archives: http://archives.free.net.ph