...

Also, please be polite. This feature was added as a convenience. Please consider using a robust crawler (instead of our simple TikaInputStream.get(new URL(fileUrl))) that will allow for better configuration of redirects, timeouts, cookies, etc.; and a robust crawler will respect robots.txt!

NOTE: In Tika 2.x, this capability has been replaced by a FileSystemFetcher. See: tika-pipes (FetchersInClassicServerEndpoints)

Transfer-Layer Compression

...