用户主页获取投稿列表 -352 新增校验 #868
Comments
|
是的，我也遇见同样问题 |
|
但是经过尝试，加了也还是有可能会有 -352 的报错，这三个字段似乎不是风控的关键内容 |
|
等待大佬 |
|
需要加User-Agent |
我添加过一下Header：
并添加了Chrome（v106）的浏览器指纹，但是依然风控，应该是别的原因 |
|
同样发现这个问题，关键是 |
|
let version;
let rendererAndVendor;
const gl = document.createElement("canvas").getContext("webgl");
if (gl) {
version = gl.getParameter(gl.VERSION);
const debugInfo = gl.getExtension("WEBGL_debug_renderer_info");
if (debugInfo) {
rendererAndVendor = gl.getParameter(debugInfo.UNMASKED_RENDERER_WEBGL) +
gl.getParameter(debugInfo.UNMASKED_VENDOR_WEBGL);
}
}
if (version === undefined) { // 禁用了 WebGL
console.log("dm_img_str", "bm8gd2ViZ2");
console.log("dm_cover_img_str", "bm8gd2ViZ2");
} else {
console.log("dm_img_str", btoa(version).slice(0, -2));
if (rendererAndVendor === undefined) { // 禁用了 WEBGL_debug_renderer_info 扩展
console.log("dm_cover_img_str", "bm8gd2ViZ2wgZXh0ZW5zaW");
} else {
console.log("dm_cover_img_str", btoa(rendererAndVendor).slice(0, -2));
}
} |
不能加Referer。 |
headers中加上cookie已暂时解决 |
|
经测试，cookie里必须要添加buvid3和buvid4。而请求url中的dm_cover_img_str为必须项，经解码后发现是GPU渲染信息，因此它为固定值，可以随便填 |
|
https://www.52pojie.cn/thread-1861424-1-1.html 这个参数已经可以解决了 |
Referer信息应该不是影响因素，我尝试过添加和移除的场景，被风控的时间是差不多的 |
|
这个我试了下,最后加了Headers加了个ua可以访问了，但是我并不确定是否频率高了仍会触发封控，需要观察。 |
|
根据这几天的实验，cookie（如： |
如 #868 (comment) 所言，你可能需要更多的实验来排除，因为单纯添加或修改某一个header或产生类似重置限频的行为 |
|
需要在headers中手动加入cookies 但是问题来了，我都手动添加cookies了，还需要爬虫再跑一次就有点徒劳无功了，请问有什么方式可以获取新的cookies呢？ |
|
|
1 use chrome privacy mode , get ua && url's dm_img_str && dm_cover_img_str ，without time testing 2 login mode use ua , cookie's buvid3 && SESSDATA ，2day live |
11.29最新补充：dm_cover_img_str现在不能随便填了，经测试后面几个字符并不是标准的base64编码，但整体依然为固定值。可以直接从浏览器里复制出来用 |
我就是直接复制的浏览器的值，这几天一直正常 |
我参考的还是这个 issue 和 #933 里的方案，具体可以看下我项目里获取动态函数里的处理方式，需要说明的是固定一个 |
前面获取w_rid和wts的代码应该会有点乱，因为我目前只会扣，这两个参数实测请求没问题，但是换个mid就不行了，我也不知道为什么 |
这个接口目前需要处理的是
|
目前我也没找到哪个地方的加密出问题，蚌埠住了 |
|
不是，你们把 cookie 给删了啊，还回复 |
|
仅针对https://api.bilibili.com/x/space/wbi/arc/search |
|
这玩意挺蛋疼的，我目前的测试是加SESSDATA和wbi，用postman能过，不加SESSDATA就-352 |
|
目前 from urllib.parse import urlencode, unquote
import requests
import hashlib
import base64
import time
import math
def md5(string: str):
return hashlib.md5(bytes(string, encoding="utf-8")).hexdigest()
def getVideoInfos(uid, startPage=1):
api = "https://api.bilibili.com/x/space/wbi/arc/search"
h = {
"authority": "api.bilibili.com",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36",
"referer": f"https://m.bilibili.com/space/{uid}",
}
timestamp = f"{math.floor(time.time())}"
params = {
"dm_cover_img_str": "QU5HTEUgKE5WSURJQSwgTlZJRElBIEdlRm9yY2UgR1QgNzMwICgweDAwMDAxMjg3KSBEaXJlY3QzRDExIHZzXzVfMCBwc181XzAsIEQzRDExKUdvb2dsZSBJbmMuIChOVklESU",
"dm_img_inter": unquote(
"%7B%22ds%22:[%7B%22t%22:2,%22c%22:%22Y2xlYXJmaXggZy1zZWFyY2ggc2VhcmNoLWNvbnRhaW5lcg%22,%22p%22:[1269,1,698],%22s%22:[101,563,618]%7D,%7B%22t%22:2,%22c%22:%22c2VjdGlvbiB2aWRlbyBsb2FkaW5nIGZ1bGwtcm93cw%22,%22p%22:[800,26,1365],%22s%22:[188,2930,1892]%7D],%22wh%22:[4183,3491,9],%22of%22:[309,618,309]%7D"
),
"dm_img_list": "[]",
"dm_img_str": "V2ViR0wgMS4wIChPcGVuR0wgRVMgMi4wIENocm9taXVtKQ",
"keyword": "玳瑁",
"index": "1",
"mid": f"{uid}",
"order": "pubdate",
"order_avoided": "true",
"platform": "web",
"pn": f"{startPage}",
"ps": "25",
"web_location": "1550101",
"wts": timestamp,
}
params["w_rid"] = md5(urlencode(params) + "ea1db124af3c7062474693fa704f4ff8")
resp = requests.get(api, params=params, headers=h)
with open("tmp/tmp2.json", "wb") as fp:
fp.write(resp.content)
if __name__ == "__main__":
getVideoInfos(547072854) |
|
在未登录情况下只需要w_rid 鼠标轨迹需要置空 页面的js走的另外一个逻辑 就不会触发风控校验了 |
举个例子： |
|
要加referer了，否则412 |
我也遇到-412，headers加referer又好了： |
早知道直接拖动到这一页的最后，直接看你的回复了。 |
|
大佬们 `from functools import reduce WRI_LIST = [ print( )` |
dm_img_list可以置空的 逻辑很简单，就一个md5 有需要可以联系我，目前测试未登录也能过风控 |
可以试下我现在的这个参数字典哪里有问题吗，感觉就是dm_img_inter这个参数编码有问题，计算与抓包不一致 |
参数引号需要转义，其他不用，已验证 |
|
未登录不太行 ...
# wbi 签名代码略
def getCookies():
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3',
'Referer': 'https://www.bilibili.com/'
}
resp = requests.get('https://api.bilibili.com/x/frontend/finger/spi', headers=headers)
resp.raise_for_status()
json_content = resp.json()
return {
'buvid3': json_content['data']['b_3'],
'buvid4': json_content['data']['b_4'],
}
img_key, sub_key = getWbiKeys()
signed_params = encWbi(
params={
'mid': '451618887',
'dm_img_list': [],
'dm_img_str': 'V2ViR0wgMS4wIChPcGVuR0wgRVMgMi4wIENocm9taXVtKQ',
'dm_cover_img_str': 'QU5HTEUgKE5WSURJQSwgTlZJRElBIEdlRm9yY2UgUlRYIDMwNjAgKDB4MDAwMDI1MDQpIERpcmVjdDNEMTEgdnNfNV8wIHBzXzVfMCwgRDNEMTEpR29vZ2xlIEluYy4gKE5WSURJQS',
# 'dm_img_inter': '{"ds":[{"t":2,"c":"Y2xlYXJmaXggZy1zZWFyY2ggc2VhcmNoLWNvbnRhaW5lcg","p":[1740,38,595],"s":[371,833,1158]},{"t":2,"c":"c2VjdGlvbiB2aWRlbyBsb2FkaW5nIGZ1bGwtcm93cw","p":[1078,52,1326],"s":[108,2848,1736]}],"wh":[5544,4023,108],"of":[208,416,208]}',
},
img_key=img_key,
sub_key=sub_key
)
query = urllib.parse.urlencode(signed_params)
cookies = getCookies()
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3',
'referer': 'https://space.bilibili.com',
'Connection': 'close',
}
url = 'https://api.bilibili.com/x/space/wbi/arc/search'
res = requests.get(url, query, headers=headers, cookies=cookies)
print(res.text)
print(res.url)仍然返回 目前的请求频率最多 30s 一次，就这样还是风控校验失败，乌鱼子 |
|
新增了参数：w_webid |
你好 刚刚找了一下 是在https://space.bilibili.com/{uid}/video 下的 <script id="__RENDER_DATA__" type="application/json"></script>中例子: <script id="__RENDER_DATA__" type="application/json"> %7 B%22 access_id%22%3 A%22 eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzcG1faWQiOiIwLjAiLCJidXZpZCI6IjMwNEI0NEI5LTU5RTQtOUExRC0wOTgxLTE0OTZFNEFFRTkxNjIxNzE3aW5mb2MiLCJ1c2VyX2FnZW50IjoiTW96aWxsYS81LjAgKFdpbmRvd3MgTlQgMTAuMDsgV2luNjQ7IHg2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEyOS4wLjAuMCBTYWZhcmkvNTM3LjM2IEVkZy8xMjkuMC4wLjAiLCJidXZpZF9mcCI6IjA2OWFkNzU4YzhmYTFmMWI3ZDJjN2VjZWUzNTk4MTQxIiwiYmlsaV90aWNrZXQiOiIwNDQyZGVjYWUxOTY0Zjk5YjM2NjA4MmZiYjQ5NWNmZSIsImNyZWF0ZWRfYXQiOjE3Mjg3MzEzOTgsInR0bCI6ODY0MDAsInVybCI6Ii8xOTIwNTA4OC92aWRlbyIsInJlc3VsdCI6Im5vcm1hbCIsImlzcyI6ImdhaWEiLCJpYXQiOjE3Mjg3MzEzOTh9.X1CBDOkZ44qFogoTQ7uiUEEwHyKnbpiUOXM0MWetRFSR9caULEl_ZONuC3GnTV0s7_LoHmHb46kgkOMv0UiLozELfmmvSdbdpAxHUdwHk1pAvly4ybPAVesI2KFKl2LLWtr-mTziuwpVuhKrLKL6-cJrN7WaLbVNZtQzrcoom71AEYOBjnG5C_LHhYolefS1yebkuot4l3ekIrK4-tgOK0JJ4uAxX2rg1_fjRVut8lfYMomf8nkKNLCYUv5LQj7xKWFNjBt2LpHK-J0zgfAe9L0IQKfYYVqXELQCiFUetm5ALrjGGJebi2vNrghsGC0unzdDCAqQymv41A614pXEmg%22%7 D</script> |
see #1107 |
事实上， 逆向+反混淆了下，定位到如下部分： {
'key': "getActiveFeaturesStr",
'value': function (_0x211a54) {
var _0x2cac63 = {
'ds': this.getElInfo(this.activityDetector.logStack, _0x211a54),
'wh': _0xb3fd69([~~this.winWidth, ~~this.winHeight]),
'of': _0x445907([~~this.scrollTop, ~~this.scrollLeft])
};
...
}
}其中
生成 点击展开 js 代码var _0x4a3c97 = {
0x9a: function (_0x5bb687) {
_0x5bb687.exports = function (_0x1f90c6, _0x31c5e5) {
if (null == _0x31c5e5 || _0x31c5e5 > _0x1f90c6.length) {
_0x31c5e5 = _0x1f90c6.length;
}
var _0x51f02c = 0x0;
for (var _0x40c03d = new Array(_0x31c5e5); _0x51f02c < _0x31c5e5; _0x51f02c++) {
_0x40c03d[_0x51f02c] = _0x1f90c6[_0x51f02c];
}
return _0x40c03d;
};
_0x5bb687.exports.__esModule = true;
_0x5bb687.exports['default'] = _0x5bb687.exports;
},
0x1ce: function (_0x9ed89f) {
_0x9ed89f.exports = function (_0x365f6e) {
if (Array.isArray(_0x365f6e)) {
return _0x365f6e;
}
};
_0x9ed89f.exports.__esModule = true;
_0x9ed89f.exports["default"] = _0x9ed89f.exports;
},
0x1c1: function (_0x37829f) {
_0x37829f.exports = function (_0x1469fb, _0x1ab30e) {
var _0x1fbf83 = null == _0x1469fb ? null : "undefined" != typeof Symbol && _0x1469fb[Symbol.iterator] || _0x1469fb['@@iterator'];
if (null != _0x1fbf83) {
var _0x17fb83;
var _0x2a83aa;
var _0x3214dc;
var _0x5f1214;
var _0xd0c4b0 = [];
var _0x322258 = true;
var _0x4d8f48 = false;
try {
_0x3214dc = (_0x1fbf83 = _0x1fbf83.call(_0x1469fb)).next;
if (0x0 === _0x1ab30e) {
if (Object(_0x1fbf83) !== _0x1fbf83) {
return;
}
_0x322258 = false;
} else {
for (; !(_0x322258 = (_0x17fb83 = _0x3214dc.call(_0x1fbf83)).done) && (_0xd0c4b0.push(_0x17fb83.value), _0xd0c4b0.length !== _0x1ab30e); _0x322258 = true) {
;
}
}
} catch (_0xf53436) {
_0x4d8f48 = true;
_0x2a83aa = _0xf53436;
} finally {
try {
if (!_0x322258 && null != _0x1fbf83["return"] && (_0x5f1214 = _0x1fbf83["return"](), Object(_0x5f1214) !== _0x5f1214)) {
return;
}
} finally {
if (_0x4d8f48) {
throw _0x2a83aa;
}
}
}
return _0xd0c4b0;
}
};
_0x37829f.exports.__esModule = true;
_0x37829f.exports["default"] = _0x37829f.exports;
},
0x1af: function (_0x570abd) {
_0x570abd.exports = function () {
throw new TypeError("Invalid attempt to destructure non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.");
};
_0x570abd.exports.__esModule = true;
_0x570abd.exports["default"] = _0x570abd.exports;
},
0x1e8: function (_0x593566, _0x1d5468, _0x8cf5d2) {
var _0x2e17e8 = _0x8cf5d2(0x1ce);
var _0x57cc6c = _0x8cf5d2(0x1c1);
var _0x5dccfd = _0x8cf5d2(0xed);
var _0x1b69a4 = _0x8cf5d2(0x1af);
_0x593566.exports = function (_0x4b3628, _0x6ee937) {
return _0x2e17e8(_0x4b3628) || _0x57cc6c(_0x4b3628, _0x6ee937) || _0x5dccfd(_0x4b3628, _0x6ee937) || _0x1b69a4();
};
_0x593566.exports.__esModule = true;
_0x593566.exports["default"] = _0x593566.exports;
},
0xed: function (_0xebb7e6, _0x44f457, _0x3cd6d2) {
var _0x36eae3 = _0x3cd6d2(0x9a);
_0xebb7e6.exports = function (_0x2f87ea, _0xbf5a3e) {
if (_0x2f87ea) {
if ("string" == typeof _0x2f87ea) {
return _0x36eae3(_0x2f87ea, _0xbf5a3e);
}
var _0xd16a61 = Object.prototype.toString.call(_0x2f87ea).slice(0x8, -0x1);
if ("Object" === _0xd16a61 && _0x2f87ea.constructor) {
_0xd16a61 = _0x2f87ea.constructor.name;
}
return 'Map' === _0xd16a61 || "Set" === _0xd16a61 ? Array.from(_0x2f87ea) : "Arguments" === _0xd16a61 || /^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(_0xd16a61) ? _0x36eae3(_0x2f87ea, _0xbf5a3e) : undefined;
}
};
_0xebb7e6.exports.__esModule = true;
_0xebb7e6.exports["default"] = _0xebb7e6.exports;
}
};
var _0x4fcdbb = {};
function _0x4c34b9(_0x1d8bf5) {
var _0xaf9ab3 = _0x4fcdbb[_0x1d8bf5];
if (undefined !== _0xaf9ab3) {
return _0xaf9ab3.exports;
}
var _0x376759 = _0x4fcdbb[_0x1d8bf5] = {
'exports': {}
};
_0x4a3c97[_0x1d8bf5](_0x376759, _0x376759.exports, _0x4c34b9);
return _0x376759.exports;
}
_0x4c34b9.n = function (_0x14ba50) {
var _0xc3b107 = _0x14ba50 && _0x14ba50.__esModule ? function () {
return _0x14ba50["default"];
} : function () {
return _0x14ba50;
};
_0x4c34b9.d(_0xc3b107, {
'a': _0xc3b107
});
return _0xc3b107;
};
_0x4c34b9.d = function (_0x4adf94, _0x2c819d) {
for (var _0x39c24a in _0x2c819d) if (_0x4c34b9.o(_0x2c819d, _0x39c24a) && !_0x4c34b9.o(_0x4adf94, _0x39c24a)) {
Object.defineProperty(_0x4adf94, _0x39c24a, {
'enumerable': true,
'get': _0x2c819d[_0x39c24a]
});
}
};
_0x4c34b9.o = function (_0x3b52d0, _0x105827) {
return Object.prototype.hasOwnProperty.call(_0x3b52d0, _0x105827);
};
var _0x613a5f = _0x4c34b9(0x1e8);
var _0x16d5d6 = _0x4c34b9.n(_0x613a5f);
var _0x5ec12e = ['g', 'w', 'A', 'Q'];
var _0x30ddcd = String.fromCharCode(0x4d);
var _0x5d32aa = String.fromCharCode(0x4e);
var _0x5773c2 = btoa(0x9b.toString()).substring(0x6);
var _0x4a678a = Number(atob(_0x30ddcd + _0x5ec12e[0x0] + _0x5773c2));
var _0x3bc098 = Number(atob(_0x30ddcd + _0x5ec12e[0x1] + _0x5773c2));
var _0x1e4c6f = Number(atob(_0x5d32aa + _0x5ec12e[0x2] + _0x5773c2));
Number(atob(_0x5d32aa + _0x5ec12e[0x3] + _0x5773c2));
var _0xb3fd69 = function (_0x361c67) {
var _0x556632 = _0x16d5d6()(_0x361c67, 0x2);
var _0x3dc4b6 = _0x556632[0x0];
var _0x2bea5e = _0x556632[0x1];
var _0x1b447a = Math.floor(0x72 * Math.random());
return [_0x4a678a * _0x3dc4b6 + _0x4a678a * _0x2bea5e + _0x3bc098 * _0x1b447a, _0x1e4c6f * _0x3dc4b6 - _0x2bea5e + _0x1b447a, _0x1b447a];
};
var _0x445907 = function (_0x61fce4) {
var _0x2fb26a = _0x16d5d6()(_0x61fce4, 0x2);
var _0x18bce7 = _0x2fb26a[0x0];
var _0x24ce6a = _0x2fb26a[0x1];
var _0x4fcb44 = Math.floor(0x202 * Math.random());
return [_0x3bc098 * _0x18bce7 + _0x4a678a * _0x24ce6a + _0x4fcb44, _0x1e4c6f * _0x18bce7 - _0x1e4c6f * _0x24ce6a + _0x4a678a * _0x4fcb44, _0x4fcb44];
};
console.log('wh:', _0xb3fd69([1920, 1080]));
console.log('of:', _0x445907([10, 10]));
输出形如: wh: [ 6093, 6631, 31 ]
of: [ 430, 760, 380 ]
关于 关于 |
and others
https://api.bilibili.com/x/space/wbi/arc/search?mid=2&ps=30&tid=0&pn=1&keyword=&order=pubdate&platform=web&web_location=1550101&order_avoided=true&dm_img_list=[]&dm_img_str=V2ViR0wgMS&dm_cover_img_str=QU5HTEUgKEludGVsLCBJbnRlbChSKSBIRCBHcmFwaGljcyBEaXJlY3QzRDExIHZzXzVfMCBwc181XzApR29vZ2xlIEluYy4gKEludGVsKQ&w_rid=e8319e347dca849eadcdd27f1a8a5a79&wts=1700567648用户投稿多了
dm_img_list、dm_img_str、dm_cover_img_str这3个值，缺少这3个值会报352错误，加上这3个值，wbi接口不添加wbi校验值都可以正常获取数据The text was updated successfully, but these errors were encountered: