Commit Graph

413 Commits

Author SHA1 Message Date
Sergey M․
86296ad2cd [utils] Add ability to control skipping false values in dict_get 2016-02-07 08:13:04 +06:00
Sergey M․
cbecc9b903 [utils] Add dict_get convenience method 2016-02-07 06:12:53 +06:00
Jaime Marquínez Ferrándiz
87de7069b9 [utils] dfxp2srt: make TTMLPElementParser inherit from object
For consistency between python 2 and 3.
2016-02-02 22:30:13 +01:00
remitamine
2b14cb566f [utils] fix dfxp2srt text extraction(fixes #8055) 2016-01-28 12:38:34 +01:00
Yen Chi Hsuan
a0d8d704df [utils] Reorder items in mimetype2ext alphabetically 2016-01-25 01:01:15 +08:00
Yen Chi Hsuan
f6861ec96f [utils] Add more items to mimetype2ext (#8293)
These are used in Youtube formats
2016-01-25 00:58:53 +08:00
remitamine
6ec6cb4e95 Revert "fix typos"
This reverts commit 36a0e46c39.
2016-01-10 19:27:22 +01:00
remitamine
36a0e46c39 fix typos 2016-01-10 17:55:41 +01:00
Jakub Wilk
dfb1b1468c Fix typos
Closes #8200.
2016-01-10 17:24:28 +01:00
Sergey M․
a7aaa39863 [utils] Extract known extensions for reuse 2016-01-04 01:08:34 +06:00
Yen Chi Hsuan
c047270c02 [utils] Remove Content-encoding from headers after decompression
With cn_verification_proxy, our http_response() is called twice, one from
PerRequestProxyHandler.proxy_open() and another from normal
YoutubeDL.urlopen(). As a result, for proxies honoring Accept-Encoding, the
following bug occurs:

$ youtube-dl -vs --cn-verification-proxy https://secure.uku.im:993 "test:letv"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-vs', '--cn-verification-proxy', 'https://secure.uku.im:993', 'test:letv']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.12.23
[debug] Git HEAD: 97f18fa
[debug] Python version 3.5.1 - Linux-4.3.3-1-ARCH-x86_64-with-arch-Arch-Linux
[debug] exe versions: ffmpeg 2.8.4, ffprobe 2.8.4, rtmpdump 2.4
[debug] Proxy map: {}
[TestURL] Test URL: http://www.letv.com/ptv/vplay/22005890.html
[Letv] 22005890: Downloading webpage
[Letv] 22005890: Downloading playJson data
ERROR: Unable to download JSON metadata: Not a gzipped file (b'{"') (caused by OSError('Not a gzipped file (b\'{"\')',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/extractor/common.py", line 330, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1886, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 471, in open
    response = meth(req, response)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/utils.py", line 773, in http_response
    raise original_ioerror
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/utils.py", line 761, in http_response
    uncompressed = io.BytesIO(gz.read())
  File "/usr/lib/python3.5/gzip.py", line 274, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.5/gzip.py", line 461, in read
    if not self._read_gzip_header():
  File "/usr/lib/python3.5/gzip.py", line 409, in _read_gzip_header
    raise OSError('Not a gzipped file (%r)' % magic)
2015-12-28 01:09:18 +08:00
Sergey M․
9b9c5355e4 Rename error_to_str to error_to_compat_str 2015-12-20 07:00:39 +06:00
Sergey M․
8e60dc7526 [utils] Add encode_compat_str 2015-12-20 06:26:26 +06:00
Sergey M․
fdae235858 [utils] Add error_to_str 2015-12-20 05:26:47 +06:00
Yen Chi Hsuan
db2fe38b55 [utils] Support alternative timestamp format in TTML
Fixes #7608
2015-12-19 19:29:51 +08:00
Yen Chi Hsuan
d631d5f9f2 [utils] Fix TTML conversion
Tolerate invalid timestamps (closes #7909)
2015-12-19 18:21:42 +08:00
Sergey M․
31b2051e21 [utils] Add remove_quotes 2015-12-14 21:30:58 +06:00
Yen Chi Hsuan
992fc9d6e1 [utils] Refactor handle_youtubedl_headers for future extension 2015-11-29 12:58:29 +08:00
Yen Chi Hsuan
0424ec307b [utils] Correct docstring of YoutubeDLHandler 2015-11-29 12:46:04 +08:00
Yen Chi Hsuan
87f0e62d94 [utils] Separate codes for handling Youtubedl-* headers 2015-11-29 12:42:50 +08:00
Sergey M․
67dda51722 Rename compat_urllib_request_Request to sanitized_Request and move to utils 2015-11-23 21:55:15 +06:00
Sergey M․
9cb9a5df77 [utils] Check ext with trailing slash against the list of known extensions 2015-11-22 17:27:13 +06:00
Sergey M․
3e12bc583a [utils] Improve determine_ext (Closes #7593) 2015-11-22 06:29:39 +06:00
Sergey M․
7e1f5447e7 [utils] Improve encode_dict 2015-11-21 20:46:33 +06:00
Sergey M․
7a3f0c00ad [utils] Style 2015-11-16 20:24:09 +06:00
Sergey M․
7aefc49c40 [utils] Skip invalid/non HTML entities (Closes #7518) 2015-11-16 20:20:16 +06:00
Jaime Marquínez Ferrándiz
6a75040278 [utils] unified_strdate: Return None if the date format can't be recognized (fixes #7340)
This issue was introduced with ae12bc3ebb, it returned 'None'.
2015-11-02 14:08:38 +01:00
Sergey M․
c90d16cf36 [utils:sanitize_path] Disallow trailing whitespace in path segment (Closes #7332) 2015-11-02 04:26:20 +06:00
Sergey M
30eecc6a04 Merge pull request #7296 from jaimeMF/xml_attrib_unicode
Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x (…
2015-10-31 18:15:21 +00:00
Sergey M․
ae12bc3ebb [utils] Make unified_strdate always return unicode string 2015-10-31 23:07:37 +06:00
Sergey M․
578c074575 [utils] Support list of xpath in xpath_element 2015-10-31 22:39:44 +06:00
Sergey M․
52c3a6e49d [utils] Improve parse_iso8601 2015-10-28 21:40:22 +06:00
Jaime Marquínez Ferrándiz
f78546272c [compat] compat_etree_fromstring: also decode the text attribute
Deletes parse_xml from utils, because it also does it.
2015-10-26 16:41:24 +01:00
Jaime Marquínez Ferrándiz
36e6f62cd0 Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x (#7178)
Attributes aren't unicode objects, so they couldn't be directly used in info_dict fields (for example '--write-description' doesn't work with bytes).
2015-10-25 20:13:16 +01:00
Sergey M․
d01949dc89 [utils:js_to_json] Fix bad escape in double quoted strings 2015-10-20 23:09:51 +06:00
Yen Chi Hsuan
1e399778ee [letv] Fix extraction
Using data URIs for passing the decrypted M3U8 manifest, which is
supported by ffmpeg only.
2015-10-18 13:42:57 +08:00
Sergey M․
af98f8ff37 [utils] Return default on fail in int_or_none 2015-10-14 22:37:03 +06:00
Sergey M․
caf80631f0 [utils] Do not fail in float_or_none on non-numeric data 2015-10-14 22:36:37 +06:00
Sergey M․
1812afb7b3 [utils] Do not fail in int_or_none on non-numeric data (Closes #7175) 2015-10-14 22:35:01 +06:00
Sergey M․
5a1a2e9454 [utils] Fix kwargs on old python 2 (Closes #6905) 2015-09-20 21:08:29 +06:00
Sergey M․
e28034c5ac [utils] Comment cookie processing until result from travis and some more testing 2015-09-06 08:16:39 +06:00
Sergey M․
266e466ee4 [utils] Simplify cookie processor 2015-09-06 07:53:11 +06:00
Sergey M․
1639282434 [utils] Add encode_dict 2015-09-06 07:22:20 +06:00
Sergey M․
ad72917274 [utils] Add issue URL in comment for #6457 2015-09-06 06:23:44 +06:00
Sergey M․
a6420bf50c [utils] Add cookie processor for cookie correction (Closes #6769) 2015-09-06 06:20:48 +06:00
Sergey M․
66e289bab4 [utils] Generalize cli option converters 2015-09-05 03:05:11 +06:00
Sergey M․
8e636da499 [utils] Improve xpath_text 2015-09-05 00:34:49 +06:00
Sergey M․
5d2354f177 [utils] Relax attribute key assert 2015-09-04 23:57:27 +06:00
Sergey M․
a41fb80ce1 [utils] Add xpath_element and xpath_attr 2015-09-04 23:56:45 +06:00
Sergey M․
e5e78797e6 [utils] Strict HTTP responses (Closes #6727) 2015-09-02 02:16:04 +06:00