JavaでEOL文字を正規化するにはどうすればよいですか？
Linuxサーバーと多くのオペレーティングシステムを備えた多くのクライアントがあります。サーバーはクライアントから入力ファイルを受け取ります。 Linuxには行末文字LFがあり、Macには行末文字CRがあり、Windowsには行末文字CR + LFがあります。
サーバーは行末にchar LFを必要とします。 Javaを使用して、ファイルが常にLinux eol char LFを使用するようにしたいと思います。どうすれば達成できますか？
2つの答えを組み合わせる（Visageとeumiroによる）：
編集：コメントを読んだ後。ライン。その場合、System.getProperty("line.separator")は使用できません。
ファイルをサーバーに送信する前に、ファイルを開いてすべてのEOLとライトバックを置き換えます
必ずそれを行うためにDataStreamsを使用し、バイナリで書き込みます
String fileString;
//..
//read from the file
//..
//for windows
fileString = fileString.replaceAll("\\r\\n", "\n");
fileString = fileString.replaceAll("\\r", "\n");
//..
//write to file in binary mode.. something like:
DataOutputStream os = new DataOutputStream(new FileOutputStream("fname.txt"));
os.write(fileString.getBytes());
//..
//send file
//..
replaceAllメソッドには2つの引数があります。最初の引数は置換する文字列で、2番目の引数は置換文字列です。しかし、最初のものは正規表現として扱われるので、'\'はそのように解釈されます。そう：
"\\r\\n" is converted to "\r\n" by Regex
"\r\n" is converted to CR+LF by Java
これやってみませんか？
content.replaceAll("\\r\\n?", "\n")
最近のプロジェクトでこれを行わなければなりませんでした。以下のメソッドは、指定されたファイルの行末を、JVMが実行されているOSによって指定された行末に正規化します。したがって、JVMがLinuxで実行されている場合、これはすべての行末をLF（\ n）に正規化します。
バッファリングされたストリームを使用するため、非常に大きなファイルでも機能します。
public static void normalizeFile(File f) {
File temp = null;
BufferedReader bufferIn = null;
BufferedWriter bufferOut = null;
try {
if(f.exists()) {
// Create a new temp file to write to
temp = new File(f.getAbsolutePath() + ".normalized");
temp.createNewFile();
// Get a stream to read from the file un-normalized file
FileInputStream fileIn = new FileInputStream(f);
DataInputStream dataIn = new DataInputStream(fileIn);
bufferIn = new BufferedReader(new InputStreamReader(dataIn));
// Get a stream to write to the normalized file
FileOutputStream fileOut = new FileOutputStream(temp);
DataOutputStream dataOut = new DataOutputStream(fileOut);
bufferOut = new BufferedWriter(new OutputStreamWriter(dataOut));
// For each line in the un-normalized file
String line;
while ((line = bufferIn.readLine()) != null) {
// Write the original line plus the operating-system dependent newline
bufferOut.write(line);
bufferOut.newLine();
}
bufferIn.close();
bufferOut.close();
// Remove the original file
f.delete();
// And rename the original file to the new one
temp.renameTo(f);
} else {
// If the file doesn't exist...
log.warn("Could not find file to open: " + f.getAbsolutePath());
}
} catch (Exception e) {
log.warn(e.getMessage(), e);
} finally {
// Clean up, temp should never exist
FileUtils.deleteQuietly(temp);
IOUtils.closeQuietly(bufferIn);
IOUtils.closeQuietly(bufferOut);
}
}
EOLの問題に対処するための包括的なヘルパークラスを次に示します。それはtyjenによって投稿されたソリューションに部分的に基づいています。
import Java.io.BufferedInputStream;
import Java.io.BufferedReader;
import Java.io.BufferedWriter;
import Java.io.DataInputStream;
import Java.io.DataOutputStream;
import Java.io.File;
import Java.io.FileInputStream;
import Java.io.FileOutputStream;
import Java.io.IOException;
import Java.io.InputStreamReader;
import Java.io.OutputStreamWriter;
import org.Apache.commons.io.FileUtils;
import org.Apache.commons.io.IOUtils;
/**
* Helper class to deal with end-of-line markers in text files.
*
* Loosely based on these examples:
* - http://stackoverflow.com/a/9456947/1084488 (cc by-sa 3.0)
* - http://svn.Apache.org/repos/asf/Tomcat/trunk/Java/org/Apache/Tomcat/buildutil/CheckEol.Java (Apache License v2.0)
*
* This file is posted here to meet the "ShareAlike" requirement of cc by-sa 3.0:
* http://stackoverflow.com/a/27930311/1084488
*
* @author Matthias Stevens
*/
public class EOLUtils
{
/**
* Unix-style end-of-line marker (LF)
*/
private static final String EOL_UNIX = "\n";
/**
* Windows-style end-of-line marker (CRLF)
*/
private static final String EOL_WINDOWS = "\r\n";
/**
* "Old Mac"-style end-of-line marker (CR)
*/
private static final String EOL_OLD_MAC = "\r";
/**
* Default end-of-line marker on current system
*/
private static final String EOL_SYSTEM_DEFAULT = System.getProperty( "line.separator" );
/**
* The support end-of-line marker modes
*/
public static enum Mode
{
/**
* Unix-style end-of-line marker ("\n")
*/
LF,
/**
* Windows-style end-of-line marker ("\r\n")
*/
CRLF,
/**
* "Old Mac"-style end-of-line marker ("\r")
*/
CR
}
/**
* The default end-of-line marker mode for the current system
*/
public static final Mode SYSTEM_DEFAULT = ( EOL_SYSTEM_DEFAULT.equals( EOL_UNIX ) ? Mode.LF : ( EOL_SYSTEM_DEFAULT
.equals( EOL_WINDOWS ) ? Mode.CRLF : ( EOL_SYSTEM_DEFAULT.equals( EOL_OLD_MAC ) ? Mode.CR : null ) ) );
static
{
// Just in case...
if ( SYSTEM_DEFAULT == null )
{
throw new IllegalStateException( "Could not determine system default end-of-line marker" );
}
}
/**
* Determines the end-of-line {@link Mode} of a text file.
*
* @param textFile the file to investigate
* @return the end-of-line {@link Mode} of the given file, or {@code null} if it could not be determined
* @throws Exception
*/
public static Mode determineEOL( File textFile )
throws Exception
{
if ( !textFile.exists() )
{
throw new IOException( "Could not find file to open: " + textFile.getAbsolutePath() );
}
FileInputStream fileIn = new FileInputStream( textFile );
BufferedInputStream bufferIn = new BufferedInputStream( fileIn );
try
{
int prev = -1;
int ch;
while ( ( ch = bufferIn.read() ) != -1 )
{
if ( ch == '\n' )
{
if ( prev == '\r' )
{
return Mode.CRLF;
}
else
{
return Mode.LF;
}
}
else if ( prev == '\r' )
{
return Mode.CR;
}
prev = ch;
}
throw new Exception( "Could not determine end-of-line marker mode" );
}
catch ( IOException ioe )
{
throw new Exception( "Could not determine end-of-line marker mode", ioe );
}
finally
{
// Clean up:
IOUtils.closeQuietly( bufferIn );
}
}
/**
* Checks whether the given text file has Windows-style (CRLF) line endings.
*
* @param textFile the file to investigate
* @return
* @throws Exception
*/
public static boolean hasWindowsEOL( File textFile )
throws Exception
{
return Mode.CRLF.equals( determineEOL( textFile ) );
}
/**
* Checks whether the given text file has Unix-style (LF) line endings.
*
* @param textFile the file to investigate
* @return
* @throws Exception
*/
public static boolean hasUnixEOL( File textFile )
throws Exception
{
return Mode.LF.equals( determineEOL( textFile ) );
}
/**
* Checks whether the given text file has "Old Mac"-style (CR) line endings.
*
* @param textFile the file to investigate
* @return
* @throws Exception
*/
public static boolean hasOldMacEOL( File textFile )
throws Exception
{
return Mode.CR.equals( determineEOL( textFile ) );
}
/**
* Checks whether the given text file has line endings that conform to the system default mode (e.g. LF on Unix).
*
* @param textFile the file to investigate
* @return
* @throws Exception
*/
public static boolean hasSystemDefaultEOL( File textFile )
throws Exception
{
return SYSTEM_DEFAULT.equals( determineEOL( textFile ) );
}
/**
* Convert the line endings in the given file to Unix-style (LF).
*
* @param textFile the file to process
* @throws IOException
*/
public static void convertToUnixEOL( File textFile )
throws IOException
{
convertLineEndings( textFile, EOL_UNIX );
}
/**
* Convert the line endings in the given file to Windows-style (CRLF).
*
* @param textFile the file to process
* @throws IOException
*/
public static void convertToWindowsEOL( File textFile )
throws IOException
{
convertLineEndings( textFile, EOL_WINDOWS );
}
/**
* Convert the line endings in the given file to "Old Mac"-style (CR).
*
* @param textFile the file to process
* @throws IOException
*/
public static void convertToOldMacEOL( File textFile )
throws IOException
{
convertLineEndings( textFile, EOL_OLD_MAC );
}
/**
* Convert the line endings in the given file to the system default mode.
*
* @param textFile the file to process
* @throws IOException
*/
public static void convertToSystemEOL( File textFile )
throws IOException
{
convertLineEndings( textFile, EOL_SYSTEM_DEFAULT );
}
/**
* Line endings conversion method.
*
* @param textFile the file to process
* @param eol the end-of-line marker to use (as a {@link String})
* @throws IOException
*/
private static void convertLineEndings( File textFile, String eol )
throws IOException
{
File temp = null;
BufferedReader bufferIn = null;
BufferedWriter bufferOut = null;
try
{
if ( textFile.exists() )
{
// Create a new temp file to write to
temp = new File( textFile.getAbsolutePath() + ".normalized" );
temp.createNewFile();
// Get a stream to read from the file un-normalized file
FileInputStream fileIn = new FileInputStream( textFile );
DataInputStream dataIn = new DataInputStream( fileIn );
bufferIn = new BufferedReader( new InputStreamReader( dataIn ) );
// Get a stream to write to the normalized file
FileOutputStream fileOut = new FileOutputStream( temp );
DataOutputStream dataOut = new DataOutputStream( fileOut );
bufferOut = new BufferedWriter( new OutputStreamWriter( dataOut ) );
// For each line in the un-normalized file
String line;
while ( ( line = bufferIn.readLine() ) != null )
{
// Write the original line plus the operating-system dependent newline
bufferOut.write( line );
bufferOut.write( eol ); // write EOL marker
}
// Close buffered reader & writer:
bufferIn.close();
bufferOut.close();
// Remove the original file
textFile.delete();
// And rename the original file to the new one
temp.renameTo( textFile );
}
else
{
// If the file doesn't exist...
throw new IOException( "Could not find file to open: " + textFile.getAbsolutePath() );
}
}
finally
{
// Clean up, temp should never exist
FileUtils.deleteQuietly( temp );
IOUtils.closeQuietly( bufferIn );
IOUtils.closeQuietly( bufferOut );
}
}
}
使用する
System.getProperty("line.separator")
（ローカル）EOL文字が表示されます。次にincomifileの分析を使用して、それがどの「フレーバー」であるかを判別し、それに応じて変換できます。
または、クライアントを標準化させてください！
パスの再帰検索で終わるファイルを変更するソリューション
package handleFileLineEnd;
import Java.io.File;
import Java.io.IOException;
import Java.nio.charset.Charset;
import Java.nio.charset.StandardCharsets;
import Java.nio.file.FileSystems;
import Java.nio.file.Files;
import Java.nio.file.OpenOption;
import Java.nio.file.Path;
import Java.nio.file.Paths;
import Java.util.List;
import Sun.awt.image.BytePackedRaster;
public class handleFileEndingMain {
static int carriageReturnTotal;
static int newLineTotal;
public static void main(String[] args) throws IOException
{
processPath("c:/temp/directories");
System.out.println("carriageReturnTotal (files have issue): " + carriageReturnTotal);
System.out.println("newLineTotal: " + newLineTotal);
}
private static void processPath(String path) throws IOException
{
File dir = new File(path);
File[] directoryListing = dir.listFiles();
if (directoryListing != null) {
for (File child : directoryListing) {
if (child.isDirectory())
processPath(child.toString());
else
checkFile(child.toString());
}
}
}
private static void checkFile(String fileName) throws IOException
{
Path path = FileSystems.getDefault().getPath(fileName);
byte[] bytes= Files.readAllBytes(path);
for (int counter=0; counter<bytes.length; counter++)
{
if (bytes[counter] == 13)
{
carriageReturnTotal = carriageReturnTotal + 1;
System.out.println(fileName);
modifyFile(fileName);
break;
}
if (bytes[counter] == 10)
{
newLineTotal = newLineTotal+ 1;
//System.out.println(fileName);
break;
}
}
}
private static void modifyFile(String fileName) throws IOException
{
Path path = Paths.get(fileName);
Charset charset = StandardCharsets.UTF_8;
String content = new String(Files.readAllBytes(path), charset);
content = content.replaceAll("\r\n", "\n");
content = content.replaceAll("\r", "\n");
Files.write(path, content.getBytes(charset));
}
}
String.replaceAll（）はコーディングが簡単ですが、正規表現インフラストラクチャを経由しないため、パフォーマンスが向上します。
/**
* Accepts a non-null string and returns the string with all end-of-lines
* normalized to a \n. This means \r\n and \r will both be normalized to \n.
* <p>
* Impl Notes: Although regex would have been easier to code, this approach
* will be more efficient since it's purpose built for this use case. Note we only
* construct a new StringBuilder and start appending to it if there are new end-of-lines
* to be normalized found in the string. If there are no end-of-lines to be replaced
* found in the string, this will simply return the input value.
* </p>
*
* @param inputValue !null, input value that may or may not contain new lines
* @return the input value that has new lines normalized
*/
static String normalizeNewLines(String inputValue){
StringBuilder stringBuilder = null;
int index = 0;
int len = inputValue.length();
while (index < len){
char c = inputValue.charAt(index);
if (c == '\r'){
if (stringBuilder == null){
stringBuilder = new StringBuilder();
// build up the string builder so it contains all the prior characters
stringBuilder.append(inputValue.substring(0, index));
}
if ((index + 1 < len) &&
inputValue.charAt(index + 1) == '\n'){
// this means we encountered a \r\n ... move index forward one more character
index++;
}
stringBuilder.append('\n');
}else{
if (stringBuilder != null){
stringBuilder.append(c);
}
}
index++;
}
return stringBuilder == null ? inputValue : stringBuilder.toString();
}
public static String normalize(String val) {
return val.replace("\r\n", "\n")
.replace("\r", "\n");
}
HTMLの場合：
public static String normalize(String val) {
return val.replace("\r\n", "<br/>")
.replace("\n", "<br/>")
.replace("\r", "<br/>");
}