padding file что это
BitComet Pollutes BitTorrent with Junk Data
So what are these padding files?
For every file in a multi-file torrent, BitComet includes a padding file by default. This overcomes the problem of ending one file and beginning another on the same BitTorrent “piece.” The feature was added to support finding sources from http/ftp/ed2k services on multi-file torrents.
Why is this a problem?
Unfortunately, BitComet’s development team sprung these padding files on to the rest of the community. If they had been more communicative, such as pre-publishing a specification, client makers and administrators of Torrent sites could then program their systems to mask them.
The padding file feature might come to bite BitComet users. It has been reported that a malicious user could create a torrent with a fake padding file. This means that Bitcomet 0.85+ users will never be able to complete their downloads without switching to another client.
Some BitTorrent users are starting to get annoyed by these (for most people) useless padding files. “Fuzzier,” who has been leading the charge on several forums (including Wikipedia), sums it up:
“I and lots of my friends don’t use BitComet, and many others stick with older versions of BitComet. We see more and more useless padding files in torrents, and it gets really inconvenient — delete them then we cannot pass hash check and cannot seed; and no matter what, we get a bunch of wastes especially our precious upload bandwidth.”
Unless this practice becomes more widely adopted in the BitTorrent community, BitComet might consider disabling this feature by default, and suppling client makers and administrators of BitTorrent sites with the specs so they can decide how to deal with them.
Паддинги
Материал из WikiTorrents
Содержание
«Padding» (англ.) — набивка, «вода», общие фразы в тексте, многословие, грунтование.
Пользователи, которые не используют BitComet, вынуждены скачивать вместе с нужными файлами и «padding_file«, что является лишней тратой времени и трафика. То есть, если пользователь скачивает коллекцию из 100 mp3-файлов, он также скачивает 100 бесполезных для него паддинг-файлов, которые в среднем занимают 3% от общего объёма раздачи (в редких случаях может доходить и до 10%).
Сам пользователь, как правило, не видит данных файлов в своей раздаче, для пользователей других клиентов задание в клиенте выглядит так:
Что такое паддинги и для чего они были придуманы?
Начиная с версии 0.85 (начало 2007 года), в BitComet добавлена возможность совмещения границ частей файлов. Это гарантирует, что один «кусочек» будет содержать в себе часть содержимого только одного файла. Также это позволяет BitComet искать и скачивать материал из других источников, таких, как HTTP / FTP-серверы или ed2k сети. Для пользователей BitComet данные файлы являются невидимыми, для остальных же клиентов они создают некоторое неудобство. Данная функция была включена по умолчанию в версии 0.85, и отключена по умолчанию в версии 0.86.
В чём заключается проблема?
К сожалению, команда разработчиков BitComet не согласовала внедрение данной функции с остальным битторент-сообществом. Если бы они были более открыты для диалога, например, предварительно опубликовали спецификацию программы, то разработчики остальных клиентов и администраторы торрент-трекеров смогли бы внести коррективы в свои программы и избежать проблем с паддинг-файлами. Но этого, увы, не произошло.
Также, использование паддинг-файлов делает пользователей BitComet уязвимыми. Возможно создание торрента с поддельными паддинг-файлами, в этом случае, пользователи Bitcomet 0.85 + не смогут завершить скачивание файлов без смены клиента.
Решение проблемы
У проблемы два пути решения.
Путь первый. Сменить установленный клиент BitComet на любой другой, рекомендованный на нашем трекере.
Путь второй. При создании торрент-файла нужно снять галочку с пункта «Выравнивать файл по размеру кусочков» (Align file to piece boundary)
A free C++ BitTorrent/HTTP/FTP Download Client
Table of Contents
Known Client Issues and Incompatibilities
Norton Antivirus constantly asks me to permit / decline things for BitComet. How do I stop this?
Solution
I’m using an Nvidia software firewall, and BitComet uses A LOT of my resources. How do I stop this?
If BitComet appears to be using a lot of your PC’s memory (or even gradually using a lot of your PC’s memory), you must uninstall Nvidia’s firewall. Disabling the firewall will be of no use, as it continues running in the background (i.e. in Processes).
This issue is not inflicted by BitComet, it is Nvidia’s firewall that is causing the problem.
PeerGuardian/Protowall Conflicts with BitComet
Please check in the logs for the next URLs:
ip-us.bitcomet.org: 221.130.193.23-221.130.193.23
ip-us.bitcomet.org: 221.130.193.33-221.130.193.33
ip-us.bitcomet.org: 221.130.195.221-221.130.195.222
ip-us.bitcomet.org: 221.130.195.224-221.130.195.226
torrent-cache.bitcomet.org: 221.130.195.227-221.130.195.227
torrent-filehash.bitcomet.org: 221.130.195.229-221.130.195.229
torrent-filehash.bitcomet.org: 221.130.195.231-221.130.195.231
inside-stats.bitcomet.com: 221.130.199.139-221.130.199.139
inside-stats.bitcomet.com: 222.73.227.222-222.73.227.222
and allow them. Your problem will be solved.
What is the 4226 issue in Windows XP SP2?
Windows XP SP2 limits the number of simultaneous TCP connection attempts to 10, at any given moment. If there are more concurrent TCP connection attempts, Windows generates a warning: “EventID 4226: TCP/IP has reached the security limit imposed on the number of concurrent TCP connect attempts”. You can go to XP’s Admin tool, Event Viewer, look in the System tab and notice tcpip entry (appears beside a yellow warning sign).
The detailed description can be found in Microsoft documentation:
In Windows versions beginning with XP SP2, there are 10 concurrent TCP connection attempts allowed simultaneously. Additional connections are placed in a queue, and will be opened no more than 10 at a time, until the target number of successful connections is reached. Windows versions prior to Windows XP SP2, did not have this limit.
Cause of this issue
The limit was added to prevent computers from unknowingly being used in a type of Denial-of-Service attack known as a “Syn Flood”, in which your computer sends an endless stream of connection requests (SYN’s) to another computer. (A SYN that has not yet been followed up on is a half-open connection.) This barrage of SYNs fills up all of that computer’s “slots” for incoming connections, so it cannot respond to anyone else until those slots are cleared.
Your computer never follows up on any of these requests, it just keeps sending more SYNs. The attacked computer gradually times-out uncompleted requests and reopens its slots, but your computer just fills up those cleared slots with more SYNs. The result is that nobody else can find an open slot to connect to on that computer.
Thus the denial-of-service.
By limiting the rate of half-open connections your computer can have pending (the number per second of SYN requests you can have that haven’t been converted to fully-open), you give the attacked computer time to clear its slots before you can re-attack them. Other computers will have a chance to find an open slot and connect.
No legitimate software application should ever behave this way for any reason, or need to start so many connections unless it’s attacking another computer. Do not attack other computers. This usually happens when you’ve been zombied and taken over by malware, because you were careless about your firewall.
This half-open limit won’t affect the speed of your torrents. Connection attempts from you will open at a steady rate of about 10 per second, until your target of successful connections is reached (which, in almost all cases, will be less than one minute). Altering this setting should only be done by expert computer users, who have a specific reason for making this change.
The Pedantic Part: A connection attempt (half-open connection) is an attempt to connect to another computer. If the TCP connection is accepted and confirmed (it’s a three-part process) (don’t ask), it is no longer “half open”, as it becomes a completed TCP connection. If there is no response or no response to the response (we said, don’t ask), the connection will remain in the “half open” state, until a time-out occurs (usually a few seconds) and the connection attempt is dropped (disconnected). The slot on the receiver becomes ready to accept another request for connection.
Event 4226 Report, when generated by normal use of BitComet, is NOT a problem and is an expected, normal occurrance.
Warning: Recent versions of BitComet include a patch which can alter your system’s tcpip.sys file, to change this rate limit. We strongly recommend this only be attempted by computer professionals, and only if there is a specific reason for wanting to change said limit, as it could cause problems, or make the system become unstable. Use this at your own risk. Don’t ask for help recovering from it, they’ll only laugh at you.
What are these “_padding_file…” things in the filelist?
When downloading a BT task where not all of the files are selected, BitComet must create a file containing boundary data. This data will be saved at present time, in a special file: taskname.piece_part.bc!
BitTorrent clients exchange data which has been broken down into small pieces, not entire original files. Often, one or more of these pieces may span over two or more files, making it necessary to download parts of a file that was not selected to be downloaded. In order for the selected files to be fully downloaded and pass a hash check, this excess data must be present, and will be stored in this additional file.
BitComet versions prior to v. 1.02 do not use this type of boundary data file (taskname.piece_part.bc!) and could, in some cases, cause valid files to fail a hash-check, if the task properties have been altered, or the files are manually hash-checked.
This file (taskname.piece_part.bc!) must be present as long as the task is active in BitComet (as well as if, in the future, you need to reseed the torrent). It’s not required, in any way however, with regards to the usage of the files after downloading.
However, prior to introducing this feature, in version 0.85 and latter, BitComet introduced the Align File to Piece Boundary option in the torrent making dialog. This option assures the torrent’s author that no single BT piece will span over two or more files, by the use of padding files. This are files created by Bitcomet, which have the exact size of the remaining unoccupied space in every piece that holds the last data chunk from the end of a file. Thus, the remaining chunk from the end of the file plus the padding file will occupy the whole piece and the padding file will end exactly on piece boundary. Since in versions v.0.85 and latter, BitComet conceals and skips these padding files, they are transparent to the end user when it looks at the contents of a torrent in BitComet. But in older versions (as well as in other clients) they are visible, hence the reason why you see and have to download these files. That is also why the advice to upgrade to the latest version appears, in order to avoid seeing and downloading them. Nevertheless you’ll still be able to download such torrents with versions older than v.0.85 or with a different client, but you’ll have to cope with the presence (annoying for some people) of the padding files in your user interface and on your disk.
For some additional details on piece boundary related issues, refer to the next topic, too.
My file download stops at 9x.x%. What can I do about that?
There may be several reasons why this happens; up to now, the most well-known ones are as following:
This problem has been solved in 0.85 and above versions. It’s suggested that you update BitComet to the latest version. Or stopping and restarting the task manually, might work.
Why do some BT tasks’ progress revert to 99.9% from 100% when their download is finished?
There are several reasons to why this happens, such as:
Torrent padding files
Did anyone created padding files for torrent? How many clients use this stuff nowadays? Is «padding files» comprehensive?
I didn’t found this feature in new clients such as ctorrent,ttorent, trasmission etc.
Do you have implementaions of this feature or some theory or history about this?
I will be appreciated for answers!
1 Answer 1
This feature was originally implemented by BitComet (with fairly poor quality of implementation in my opinion) as a way to make it simpler to de-select certain files from being downloaded. Since bittorrent downloads pieces, and pieces may span multiple files. Otherwise when you de-select some files you may still get the first and last bit of it (because it overlaps pieces that you need for other files).
Introducing pad-files ensures that files are aligned to pieces and make this problem go away.
Notably, uTorrent instead would stick these left-over pieces in a separate file, called a part file.
In more modern times, a couple of years later, libtorrent and uTorrent got some support for pad files, for a different reason. Primarily these two:
uTorrent implemented support for «mutable torrents», the ability to replace a torrent with a new version and efficiently transition/duplicate all common content to the new torrent. In order to efficiently use this feature at scale, you have to align large files to pieces to avoid having to re-hash all content (i.e. you only want to hash the new content, not the content that stayed the same). For this reason, pad files were useful.
Accessing the filesystem at cluster aligned offsets within files is potentially a lot cheaper than unaligned access. It also allows use of some more sophisticated APIs that may restrict file offsets (async. I/O). This is also true for memory mapped files.
There are more elegant ways of implementing pad files, and I believe libtorrent and uTorrent do it better. For instance, you can consolidate all pad files in a (hidden) directory when creating the torrent. uTorrent will also put partial pad files in its partfile.
As for clients supporting pad files, here are some I can think of off the top of my head (given sufficiently recent version):
Please ignore the padding files inserted by BitComet Client in torrent. #579
Comments
kirbyzhou commented Apr 15, 2018
BitComet will insert dummy padding files with zero-filled content into its torrent files.
These files are useless, and their filenames start with «____padding_file«.
Smart BT Client such as Thunder/XunLei will ignore these files while downloading.
The text was updated successfully, but these errors were encountered:
DanielYWoo commented Oct 5, 2018
ghost commented Apr 14, 2019
This is really more of a bug in BitComet.
DanielYWoo commented Apr 15, 2019
@ymte yes, BitComet hacks the protocol and generates those padding files, it sucks, but I still hope transmission can ignore them.
Auska commented Jul 19, 2019
fjqingyou commented May 13, 2020
hope add a global ignore file,
user can config it
kirbyzhou commented May 14, 2020
hope add a global ignore file,
user can config it
chrysn commented May 27, 2020
Just ignoring them will not give all the benefits a padding file has (which is to allow deduplication over different torrents / torrent versions with some identical files without having to actually transfer the padding over the wire), and would require user intervention that I’d expect rarely to be accurate in practice.
There’s three aspects to it that could largely be addressed individually: