Hello
I have to read large text-files (> 100 MB) into memory, scan the content and update a database.
I use TFileStream to read the data, then copying the filestream to a stringstream for better string handling.
If I seek a substring in Stringstream.DataString, I get an "Out-of-memory-Exception" if the size of the Stream exceeds 200 MB (it worked well with a size <= 136 MB and crashed with a size >= 280 MB).
I'm working with D2009, Windows 7 (32bit and 64bit), Windows-Server 2008 R2

Hi,
  (snip)

In Delphi 2007 you get no exception because there you only reference the
  string whereas in the unicode version (delphi 2009 and up) the things
  got complicated and they had to copy the string using the
  right encoding (I guess due to the byte oriented streaming mechanisms

(snip)

Your text file is converted to unicode and it holds twice as data now. since you were
  using Delphi 2007 before, I suspect that your text file is not unicode.
  Instead of TStringStream try this class (I haven't fully tested it but it should work)
  {code}
  TRawStringStream = class(TBytesStream)

(snip)

Actually, it would be the best if you skip using any kind of string stream and load
  string directly.
  {code}
  function RawStringLoadFromStream(Stream: TStream; Size: Integer = -1): RawByteString;
  begin

(snip)

Dear Dalija
    Thank you very much for resolving the problem with this elegant function.
    Kind regards
    Samuel
    

wrote in message news:✉forums.embarcadero.com...
    (snip)

There are three problems with that code:
    1) you are using RawByteString, but you are not specifying any codepage for
    it.
    2) you need to use TStream.ReadBuffer() instead of TStream.Read() to ensure

(snip)

Hello Mike
    Thank you for the hint, I was not aware of the DBCS-encoding.
    I'll try another approach.
    
    

wrote in message news:✉forums.embarcadero.com...
  (snip)

In pre-D2009 versions, TStringStream derived directly from TStream, used an
  AnsiString as its data storage (hense its name), and the DataString property
  provided direct access to that data.
  In D2009 and later, TStringStream now derives from TByteStream, thus uses a

(snip)

Thanks!
  Dalija Prasnikar