What is the "Turbo Download Manager (3rd edition)" extension?

This is the third iteration of the "Turbo Download Manager (TDM)" extension, which relies heavily on newly introduced ES6 methods like the streaming fetch API and asynchronous calls. This extension can download a single file concurrently through multiple threads. The segmented binary data is temporarily stored within the internal IndexedDB storage. In theory, as each thread functions as an independent downloader, the download speed increases by having more threads. Once the entire file has been fetched from the server, these segments are merged, and the file is saved to your browser's default download directory. The extension employs native APIs to transfer the segmented data from internal storage to the default download directory, ensuring that your browser maintains a record of the download, similar to a standard download initiated by your browser's default download manager.

This extension can read data from the server and write it to internal storage at speeds of up to 80 megabytes per second. If, for any reason, you have access to a faster network, it is recommended to use your browser's default download manager. Thanks to the extension's chunked data storage, it maintains an acceptable memory footprint even during large file downloading.

What happens if my browser crashes or the server becomes unreachable during a download?

In such cases, this extension employs a chunked data storage method. If a crash occurs, the binary data remains accessible in the internal IndexedDB storage, and the extension automatically restores the data upon a restart. When dealing with network issues, the extension attempts to reconnect to the download server multiple times (a configurable setting). The job enters the interrupted state when no active thread receives data. However, there's no need to worry as you can manually trigger a retry using the provided retry button. Starting from version 0.5.3, the extension retains fetched data unless the user presses the remove button. This means you can retry the download as many times as needed, even on an unstable network, until you complete the download job.

What are the default and recommended thread settings for this extension?

By default, this extension employs three threads for each download task, which is the recommended setting, and it can enhance download speeds by up to threefold. However, there are scenarios where a server lacks support for range-based downloading, making it impossible to use multiple threads. In such instances, the extension resorts to the built-in download manager to retrieve the desired file. It's important to note that each thread functions as an independent task, so while increasing the number of threads can boost download speed, it also escalates disk access and memory allocation. In its default configuration, this extension uses three threads, each one downloading up to 100 megabytes of data. Consequently, the extension consumes approximately 300 megabytes of memory for each active download job. If a user requests to download a task with, for instance, eight threads and the file is sufficiently large, memory usage could reach around 800 megabytes. Therefore, opt for higher thread counts only if memory allocation isn't a concern for your system.

Why does the interface no longer display the size of downloaded segments when a job is restored after a crash or restart?

Upon restarting your browser, this extension examines the internal storage for incomplete jobs. If such a job is found, only its metadata is retrieved from the storage and nothing more. This is done to prevent excessive disk access during startup. Consequently, the interface is unable to calculate and show the downloaded segments at this stage. However, when the user decides to resume the job, the extension conducts a thorough scan of the internal storage, reconstructs the segments, and subsequently fetches only the remaining segments needed to complete the download.

What is the meaning of the "store" button in the context menu and the "add a new job" interface?

The "store" feature is a recent addition introduced in the v3 edition of the extension. It leverages your browser's synced storage to save links you intend to download later. You can conveniently add multiple links to this list, and subsequently download them on any device synchronized with your current one. To access all the stored jobs, open the extension's popup interface. It's important to note that when a job from this list is sent to the downloader for processing, it is removed from the synced storage.

How do I go about downloading media files from the current page?

In the v3 edition of the extension (v0.5.3 or later) of this extension, when a media file is identified on a page (usually when it's about to play, triggering an event that prompts the download manager to gather the media link), you'll notice a change in the color of your browser's action button. To view the collection of gathered media links, you can use the "Extract Media Links" option in the page context menu. Alternatively, you can open the toolbar popup and click the "Add new Job(s)" button, which will display a list of all the downloadable media links on the page.

Is it possible to use this extension for downloading all segments of an M3U8 stream?

Starting from version 0.5.6, the "Add Jobs" view has been enhanced to support the parsing of M3U8 streams. When you add a new M3U8 link, the extension automatically retrieves its content and parses it. In cases where the manifest file contains multiple playlists, you'll be prompted to select the desired playlist. If file segments are identified within the playlist, the extension will indicate the total number of segments to be downloaded and merged. It's important to note that M3U8 parsing functionality is exclusively available within the "Add Jobs" view. Therefore, be sure to add your M3U8 jobs through this view rather than directly.

It is recommended to check the "Live Stream Downloader" extension, which is a useful tool for all kinds of streaming tasks.

What does the "Merge and Download" button in the "Add Jobs" view do?

This button treats all the links within the "Download List" as individual segments of a unified file. The extension downloads each link and appends its binary content to the end of the preceding link until all links are completely downloaded. This function is useful to consolidate chunked data into a single cohesive file. It's essential to ensure that only segmented links are present in the "Download List" section and that they are arranged in the correct order. Use the M3U8 feature (when possible) rather than relying on this particular function for downloading segmented streams.

I've noticed that in v0.5.6, the extension's capability to detect media streams on web pages is not as robust as in previous versions. How can I address this issue?

This extension has introduced an optional feature that uses the web request API to monitor web activities and provide users with media links for each page starting from version 0.5.6. By default, this feature is inactive. To enable it, you should visit the options page and grant the extension permission to use this feature. However, even without this permission, the extension can still capture media streams when they are played. Activating this feature, however, can enhance the detection of media streams.

I have a local M3U8 file, and I want TDM to download the links from it as a single binary file. However, TDM doesn't support adding local files. Is there a way to instruct the extension to download this list?

Open the M3U8 file using a text editor and copy all the links to your clipboard. Next, open the "Add Jobs" view in a browser tab that hasn't detected any jobs. The extension will automatically extract the links from your clipboard and add them to the "Download List." Take a moment to manually review the list and remove any entries that are not part of the M3U8 file. Finally, click the "Merge and Download" button. This button treats all the links as segments of a single binary file, allowing you to download them as a cohesive whole.

It is not recommended to play with the internal preferences. However, it is doable. Here is the list of all configurations that can alter in edition 3 version 0.5.3:

• absolute-max-segment-size: The maximum range size of each thread. When a range is calculated it is always smaller than this size. The value is in bytes and the default value is 100MBytes.
• max-number-memory-chunks: When chunked data is received from a server, it is stored in memory until the disk is ready for writing the binary data. If the size of the memory array increases from this value, all the network activities are blocked (to prevent more memory allocations) until the disk writing is finished and the disk state is ready. The default value is 500 which means up to 500 binary objects can be stored in the memory. Note that in case of a crash, this data will be lost.
• max-number-of-threads: This is the maximum simultaneous number of threads that a downloading job can have. The default value is 3. When a thread is finished the downloader checks this value and decides whether a new thread should be added or not.
• max-retires: This is the maximum number of sequential errors that can occur without any byte being fetched from the server before the downloading state changes to the "interrupted" state. The default value is 10.
• max-segment-size: This is the size of each thread (range) by default. The default value is 100MBytes. Note that for a small file, the range size can be smaller. But for a large file, the range size is this value unless it is altered later by "overwrite-segment-size".
• max-simultaneous-writes: This is the maximum simultaneous IndexedDB write handles that the extension uses for a job. The default value is 3. This means the extension uses up to 3 write handles to write the memory chunks to the IndexedDB database.
• min-segment-size: This is the minimum size of a range in bytes. The default value is 100KBytes. So if a file is for instance 200KByes and the number of threads is set to 3, because of this value, only 2 threads are created.
• overwrite-segment-size: if true, the "max-segment-size" configuration is determined by file size divided by "max-number-of-threads" after headers are resolved.
• speed-over-seconds: Number of seconds to be used to calculate the current downloading speed. The default is 10 seconds.
• use-native-when-possible: If a downloading server does not support multi-threading, when this option is true, the native download manager handles the job instead.
• max-file-size: This is the maximum filesize that the extension allows multi-threading. Files larger than this value will be sent to the native download manager.

chrome.storage.local.set({
  'internal-get-configs': {
    'absolute-max-segment-size': 100 * 1024 * 1024
  }
});

I am a developer and I would like to send jobs to this extension. Is it possible?

As of version 0.5.3, this extension supports receiving jobs from other extensions in the following format:

chrome.runtime.sendMessage(THS_EXTENSION_ID, {
  method: 'add-jobs',
  jobs: [{
    link: 'http://www.example.com',
    threads: 3
  }]
}, resp => {
  if (!resp) {
    console.log('The TDM is not installed');
  }
});

chrome.runtime.sendMessage(THS_EXTENSION_ID, {
  method: 'store-links',
  links: ['http://www.example.com']
}, resp => {
  if (!resp) {
    console.log('The TDM is not installed');
  }
});

[deprecated] What does the "Turbo Download Manager (v2)" extension entail?

The "Turbo Download Manager (v2)" extension is essentially a WebExtension adaptation of my previous "Turbo Download Manager" Firefox extension (or Chrome app).

This extension comprises four primary modules:

1. Download Manager Popup: This feature serves as a monitoring hub for ongoing download tasks, whether initiated from this extension or through the built-in download manager.

2. Download Dialog Window: The download dialog window is designed for adding multiple links and gathering images from the opener tab, facilitating efficient job management.

3. Multi-Threading Internal Module (downloads.js): This module is responsible for handling multi-threading tasks, and enhancing download speed.

4. Download Observer Module: This module keeps tabs on video and audio sources requested by each webpage.

To monitor ongoing downloads, click on the toolbar button to open the popup. Within this popup, you can pause, resume, and view the number of segments for multi-threading tasks. It's important to note that this is the exclusive location where you can monitor and halt multi-threading jobs. Multi-threading jobs do not appear in your browser's default download manager until they are fully fetched.

To initiate new download tasks, you must open the "Download dialog window." Access to this window can be obtained either from the "Download Manager popup" or by right-clicking the toolbar button. This window facilitates the simultaneous initiation of multiple downloads with a customizable number of threads. Please keep in mind that if you select a single thread for a download, this extension redirects the request to the internal download manager, as there is no advantage to obtaining a single-threaded file using the internal "downloads.js" module. To initiate a file with the built-in download manager, reduce the number of threads to one. Additionally, you can define default thread sizes for image, audio, and video formats via the options page.

How does the multi-threading process operate?

When a new download job is initiated, the extension examines the server to verify if it supports multi-threading. If multi-threading is not supported, your download is managed by your browser's built-in download manager. However, if the server does support multi-threading, the extension proceeds as follows:

1. It calculates the total size of the file to be downloaded.
2. The file is then divided into various segments.
3. The extension requests your browser to allocate the necessary disk space locally.
4. If sufficient space is available on your system, all segments are initiated one by one.
5. Once all segments are successfully downloaded, the file is transferred from internal storage to your user-defined Downloads directory. This transfer is handled by your browser's built-in download manager, ensuring that the download is tracked alongside normal downloads.

It's important to note that, unlike the built-in download manager, the extension doesn't create temporary files in the Downloads directory because it lacks access to that directory. Instead, it utilizes internal storage for this purpose.

A note for Firefox users: As of the time of writing this documentation, the Firefox browser does not yet support the "FileSystem" APIs required for file creation and segmented writing, which are essential for multi-threading. The extension includes a wrapper module using "IndexedDB" APIs as an alternative, but this method is not as efficient. Once Firefox supports "FileSystem" APIs, the extension will automatically use them, enabling the multi-threading functionality.

What's new in this version?

Please check the Logs section.

What's the process for creating a new download task?

You have two methods to add a new download job:

1. Using the Right-Click Context Menu: You can right-click on a link, video, audio, or image element and select the "Add New Job" option from the context menu. This action will initiate a download task. Please note that when using this method, the default thread count is determined based on the mime type of the link.

2. Using the "Download Dialog Window": Alternatively, you can add a download by opening the "Download Dialog Window." You can access this window by right-clicking the action button and selecting the appropriate option. When you use the "Download Dialog Window" to add a download, you can individually customize the number of threads for each job.

[deprecated] How can I download all the images from a webpage?

Follow these steps: 1. Navigate to the tab containing the webpage from which you want to collect images. 2. Next, access the "Download Dialog Window" by right-clicking the toolbar button and selecting the appropriate option. 3. Within the user interface, you'll find a button that allows you to search for images loaded in the opener tab. Click this button and wait for the extension to gather all the images. 4. Once the extension has collected all the images, you'll see a list in the user interface. 5. Select the images you wish to download from the list, and then click the "Download" button to initiate the download process.

[deprecated] How can I download all media files, including videos and audio on a webpage?

This extension uses an internal module designed to track all media sources on a webpage. You can conveniently monitor the number of detected media sources via the badge displayed on the toolbar button for the current tab. To access and download these media sources, follow these steps:

1. If the webpage you're on has media sources, you'll see the badge number on the extension's toolbar button.

2. Open the "Download Dialog Window" while you're on a page with the badge number. This window will list all the detected media sources.

3. In case a webpage has multiple media sources, and you're unsure how to obtain the last media link, begin by right-clicking on the toolbar button and selecting the "Reset" option.

4. Now, initiate the tracking process for the specific media source you want access to. When the toolbar button's badge text updates, indicating the presence of media sources, proceed to open the "Download Dialog Window."

[deprecated] What is the meaning of the badge text on the toolbar button?

The badge text on the toolbar indicates the count of audio and video sources being used on the current page, whether they are actively being played or cached. If you wish to reset this count, you can use the right-click context menu option. To view the list of these detected sources, open the "Download dialog window." Within this window, there is also an option to copy all the identified links to your clipboard for easy access.

How can I keep track of my download tasks?

To monitor your download jobs, you can simply left-click on the toolbar button, which will open the "Download Manager Popup" view. Within this view, you'll find a list that displays all your downloaded files as well as those currently in progress. It's worth noting that this is the sole location where you can oversee multi-threading jobs in addition to the built-in ones.

What's the suggested thread count?

For smaller files, it's advisable to use two threads, while for larger files, opting for three threads is recommended.

I unintentionally added a large list of downloads to the extension by mistake. Is there a way to remove all of them in one go?

Here's how you can do it:

1. Access your browser's extensions manager window.
2. Restart the TDM extension to halt all ongoing jobs.
3. Open the extension's popup.
4. While holding down the Ctrl key on Windows or the Command key on Mac, left-click the "Failed" button (you can hover your mouse over this button to view the instructions). This action will effectively erase all the paused downloads in one step.

[user question] When I attempt to download a large file, approximately 2 GB in size, I notice that it takes roughly 5 minutes for the extension to acquire the file, followed by an additional 20 minutes for merging and saving it to my local disk. What might be causing this delay?

This extension utilizes an IndexedDB database to store file chunks until the entire file is retrieved. Once all segments are obtained, the extension proceeds to merge them to reconstruct the binary file, which is then transferred to your local disk. This method offers two significant advantages:

Firstly, it ensures that all chunks are promptly stored on your local disk as soon as they are fetched, preventing potential memory-related issues.

Secondly, it allows for the simultaneous downloading of different segments from various parts of a single file.

However, the downside of this method is the time it takes for the extension to copy the binary data from the internal IndexedDB to the local disk. The duration of this process depends on your storage speed and how efficiently the browser handles exporting a substantial amount of data from the internal IndexedDB database.

For most users, the time taken for a gigabyte-sized file is generally less than a minute, which is typically acceptable. Nevertheless, if you find that the merging process takes longer than desired, you have the option to switch to single-threading mode. Alternatively, use the "Live Stream Downloader" extension. This extension writes directly to the user disk.