User:Atvelonis/Bot

A list of bot tasks for KINMUNE or AkulakhanBot. If I don't get to your request immediately, it's most likely because the bot is occupied with another task, but don't let that stop you from adding any more that you have. Some of the larger projects for the bot, namely semi-automatic ones, can take a week or more to complete. However, most regular tasks can be finished in less than a day.

Usage
Entries on this page are likely to be carried out by one of two active bots on the wiki: AkulakhanBot, in use since May 2017, is operated by Atvelonis and makes menial edits via AutoWikiBrowser or sometimes custom Python scripts. KINMUNE, in use since February 2014, is operated by Flightmare and is typically responsible for somewhat more advanced tasks via custom Python scripts.

AutoWikiBrowser
AutoWikiBrowser (AWB) is used by AkulakhanBot to complete simple "find & replace" tasks, such as link fixes and basic formatting. A bot cannot do anything particularly advanced with this interface, but it is still invaluable for general wiki maintenance.

Examples of tasks that would be completed with AWB:
 * Skyrim → Skyrim (Online) for links on Online articles
 * n/a → N/A in tables and infoboxes
 * Removing specific overarching categories

Custom scripts
More advanced (but still menial) tasks will generally be done by writing custom Python scripts instead of using AutoWikiBrowser: this may take a while if a new script is required, depending on the urgency of the task.

Examples of tasks would be completed with custom scripts:
 * Automatically logging public messages in the Chat
 * Nested template removal
 * Portable Infobox gallery fix

In progress
The bot is either currently working on these things, or has just finished them.
 * OnlineCreatures/DLC:
 * Remove "magic," "stamina," "armor," "spells," and "damage"
 * Add "attacks."
 * TES:Moot/August 2018 ESO/DLC infobox merges
 * Check that dlc is on each page that uses an ESO infobox (final step) (see: Category:Infobox Templates/Online)
 * OnlineFactions
 * Add variable for base
 * Add variable for DLCs and change template
 * Imperial City
 * Orsinium
 * Thieves Guild
 * Dark Brotherhood
 * ESO Morrowind
 * Clockwork City
 * Summerset
 * Murkmire
 * OnlineItems
 * Add variable for base
 * Add variable for DLCs and change template
 * Imperial City
 * Orsinium
 * Thieves Guild
 * Dark Brotherhood
 * ESO Morrowind
 * Clockwork City
 * Summerset
 * Murkmire
 * OnlineLocations
 * Add variable for base
 * Add variable for DLCs and change template
 * Imperial City
 * Orsinium
 * Thieves Guild
 * Dark Brotherhood
 * ESO Morrowind
 * Clockwork City
 * Summerset
 * Murkmire
 * OnlineQuests
 * Add variable for base
 * Add variable for DLCs and change template
 * Imperial City
 * Orsinium
 * Thieves Guild
 * Dark Brotherhood
 * ESO Morrowind
 * Clockwork City
 * Summerset
 * Murkmire

High priority
If you want me to do something quickly, place it under the "high priority" section (please do not place it in a lower tier just to be polite). I will get to it as fast as I can.
 * OnlineQuests/DLC:
 * Add faction classification in "type" parameter for ESO quests (& add cat; remove side quests cat)
 * Add "title" section to LegendsCharacter template.
 * Remove the subcategories of Category:Morrowind: Characters by Location per TES:Moot/September 2018.
 * Category:Morrowind: Abaelun Mine Characters
 * Category:Morrowind: Abanabi Characters
 * Category:Morrowind: Abassel's Yurt Characters
 * Category:Morrowind: Abebaal Egg Mine Characters
 * Category:Morrowind: Abernanit Characters
 * Category:Morrowind: Adanumuran Characters
 * Category:Morrowind: Addadshashanammu Characters
 * Category:Morrowind: Addamasartus Characters
 * Category:Morrowind: Aharasaplit Camp Characters
 * Category:Morrowind: Aharnabi Characters
 * Category:Morrowind: Aharunartus Characters
 * Category:Morrowind: Ahinipalit Characters
 * Category:Morrowind: Aidanat Camp Members
 * Category:Morrowind: Ainat Characters
 * Category:Morrowind: Almurbalarammi Characters
 * Category:Morrowind: Ansi Characters
 * Category:Morrowind: Arano Plantation Members
 * Category:Morrowind: Arenim Manor Characters
 * Category:Morrowind: Argonian Mission Characters
 * Category:Morrowind: Arkngthand Characters
 * Category:Morrowind: Arobar Manor Characters
 * Category:Morrowind: Arrille's Tradehouse Characters
 * Category:Morrowind: Asha-Ahhe Egg Mine Characters
 * Category:Morrowind: Ashalmimilkala Characters
 * Category:Morrowind: Ashamanu Camp Characters
 * Category:Morrowind: Ashanammu Characters
 * Category:Morrowind: Ashinabi Characters
 * Category:Morrowind: Ashir-Dan Characters
 * Category:Morrowind: Ashunartes Characters
 * Category:Morrowind: Assalkushalit Characters
 * Category:Morrowind: Assarnatamat Characters
 * Category:Morrowind: Assarnud Characters
 * Category:Morrowind: Assernerairan Characters
 * Category:Morrowind: Assu Characters
 * Category:Morrowind: Assumanu Characters
 * Category:Morrowind: Assurdirapal Characters
 * Category:Morrowind: Beshara Characters
 * Category:Morrowind: Black Shalk Cornerclub Characters
 * Category:Morrowind: Cavern of the Incarnate Characters
 * Category:Morrowind: Esutanamus Characters
 * Category:Morrowind: Gro-Bagrat Plantation Characters
 * Category:Morrowind: Habinbaes Characters
 * Category:Morrowind: Hinnabi Characters
 * Category:Morrowind: Indarys Manor Characters
 * Category:Morrowind: Kudanat Characters
 * Category:Morrowind: Minabi Characters
 * Category:Morrowind: Panat Characters
 * Category:Morrowind: Pinsun Characters
 * Category:Morrowind: Saturan Characters
 * Category:Morrowind: Sha-Adnius Characters
 * Category:Morrowind: Shushan Characters
 * Category:Morrowind: Shushishi Characters
 * Category:Morrowind: Sinsibadon Characters
 * Category:Morrowind: Tel Uvirith Characters
 * Category:Morrowind: Ules Manor Characters
 * Category:Morrowind: Yakanalit Characters
 * Category:Morrowind: Zainsipilu Characters
 * Category:Morrowind: Zebabi Characters
 * TES:Moot/January 2019: 3 and 4 (2 also needs to be done)
 * TES:Moot/March 2019: 4 (7 also needs to be done)

Low priority
If your task does not have any sort of deadline or is otherwise not very urgent, place it in the "low priority" section.
 * Category:Articles to be Moved
 * Add LE to achievement names on Achievements (Online)
 * Remove space between "Base ID" in infoboxes
 * Add ==Licensing== header to files missing it (excluding those where it is transcluded in image licensing template)
 * Transition all written categories to image licensing templates for consistency
 * Remove extraneous licensing content for files using Category:Image Licensing Templates
 * creatures =, type= ->
 * Add |thumb to files with captions in blogs and elsewhere (without this, the captions will not appear)
 * Do NOT replace center/left/right; just add it as the second parameter, right after the file name
 * Replace "NPCs" with "characters."
 * Add "previous" and "next" variables to pages using OnlineBooks and DLC derivatives
 * Remove overarching merchant categories for ESO NPCs (DLCs only? Check!)
 * Remove the Category:ESO Morrowind: Enemies from pages
 * Remove the Category:Morrowind: Enemies from pages
 * Remove the Category:Arena: Enemies from pages
 * Remove the Category:Online: Enemies from pages
 * Change Ice Wolf to Ice Wolf (Skyrim), fix links from other articles
 * Replace the |enemies parameter in Template:OnlineLocations and all the Online locations with |creatures. If something is written there, it should be moved to the |characters or to the |creatures parameters

Content
For semi-automatic tasks that require me to add content. Such tasks may take a long time to finish.
 * Add ESO NPC classes (where applicable)
 * Category: Online: Characters (+DLC); skip if contains "class = N/A" (include additional whitespace)
 * Remove enemy stuff for all games; replace with either character or creature, depending on context
 * & add hostility variables etc. for NPCs/creatures, per TES:Moot/June 2018
 * Missing (locate and add info for each) – Regex for spacing
 * race =
 * gender =
 * MorrowindCharacters (Regex: skip if includes RefID)
 * Add book header content to ==Content== section. e.g. title/author, if it's in the book itself

Custom tasks

 * Sort interwiki links alphabetically

Custom scripts
For transparency. Scripts can easily be scheduled on Windows 10 with the Task Scheduler. Credit to Flightmare for almost all of this (see also). Note that some of the code has been simplified from Flightmare's original code because I haven't bothered to implement the real thing just yet (this is why the account credentials are hardcoded, for example).


 * Preliminary  script for login (original)

import requests import os import json import time
 * 1) !/usr/bin/env python3
 * 2) Credit: Flightmare

headers = {'Connection': 'Keep alive', 'Content-Type': 'application/x-www-form-urlencoded', 'User-Agent': 'Atvelonis/Bot'}

def login(wiki, username, password): session = requests.Session session.post('https://www.wikia.com/logout', data={}, headers=headers) # Logout just in case time.sleep(1) # For ratelimit print(is_logged_in(session, username, wiki)) r = session.post('https://services.fandom.com/auth/token', data={   	'username': username,    	'password': password	}, headers=headers) print(r.json) # TODO: test for login failures return session

def is_logged_in(session, username, wiki): payload = {'action': 'query', 'meta': 'userinfo', 'format': 'json'} r = session.get('https://community.wikia.com/api.php', params=payload, headers=headers) time.sleep(1) # For ratelimit return r.json['query']['userinfo']['name'] == username
 * 1) bool: true if logged in as provided user, false for other user or anon

def get_edit_token(session, wiki): payload = {'action': 'query', 'prop': 'info', 'intoken': 'edit', 'titles': '#', 'format': 'json'} r = session.post('https://community.wikia.com/api.php', data=payload, headers=headers) time.sleep(1) # For ratelimit return r.json['query']['pages']['-1']['edittoken']

def get_wiki_id(session, wiki): payload = {'action': 'query', 'meta': 'siteinfo', 'siprop': 'wikidesc', 'format': 'json'} r = session.get('https://'+wiki+'.wikia.com/api.php', params=payload, headers=headers) time.sleep(1) # For ratelimit return r.json['query']['wikidesc']['id']


 * script to replace the entire contents of a file summary (original)

''Warning: bad programming by Atvelonis. Loop + recursion is silly. :P''

import core import json import requests import time """ core.py: for login json: https://docs.python.org/3/library/json.html requests (cmd: 'py -m pip install requests'): http://docs.python-requests.org/en/master/ time: https://docs.python.org/3/library/time.html """
 * 1) !/usr/bin/env python3

headers = {"Connection": "Keep alive", "Content-Type": "application/x-www-form-urlencoded", "User-Agent": "Atvelonis/Bot"}
 * 1) Info for the web server. https://en.wikipedia.org/wiki/HTTP_persistent_connection
 * 2) The API for Discussions asks for Content-Type, User-Agent. Implemented here for consistency with non-editing scripts

wiki = "elderscrolls" username = "AkulakhanBot" password = ""
 * 1) Account credentials for the bot to log in

session = core.login(wiki, username, password) print(core.is_logged_in(session, username, wiki)) # Boolean: check if logged in or not wiki_id = core.get_wiki_id(session, wiki) edit_token = core.get_edit_token(session, wiki) print(edit_token) # https://www.mediawiki.org/wiki/Manual:Edit_token
 * 1) Calls on functions in core to log in and begin edit session

def file_licensing(startImage): # Uses the "action=query" module to generate a list of images. 5000 is the max for sysops/bots in MediaWiki # https://www.mediawiki.org/wiki/API:Query # https://www.mediawiki.org/wiki/API:Allimages payload = {"action": "query", "list": "allimages", "aifrom": startImage, "ailimit": "5000", "format": "json"} decoded_json = session.get("https://"+wiki+".fandom.com/api.php", params=payload, headers=headers).json # print(decoded_json)

for page in decoded_json["query"]["allimages"]: # Uses the "action=raw" module to returns a page's wikitext payload = {"action": "raw"} body = session.get("https://"+wiki+".fandom.com/wiki/"+page["title"], params=payload).text time.sleep(0.5) # For ratelimit

# Changes the desired content if not "{{Imagequality" in body and not "{{Information" in body: body = "{{Information\n|attention [...]"

# Publishes the edit. https://www.mediawiki.org/wiki/API:Edit payload = {"action": "edit", "title": page["title"], "summary": "Adding full file licensing", "bot": "1", "watchlist": "nochange", "format": "json", "text": body, "token": edit_token} print(session.post("https://"+wiki+".fandom.com/api.php", data=payload, headers=headers).text) print(body)

# Recursive call so that loop continues for query > 5000 (next page in list) # https://www.mediawiki.org/wiki/API:Raw_query_continue if "query-continue" in decoded_json: return file_licensing(decoded_json["query-continue"]["allimages"]["aifrom"])

file_licensing("")


 * Generic  script (original)

import core import json import requests import time import re """ core.py: for login json: https://docs.python.org/3/library/json.html requests (cmd: 'py -m pip install requests'): http://docs.python-requests.org/en/master/ time: https://docs.python.org/3/library/time.html re: https://docs.python.org/3/library/re.html """
 * 1) !/usr/bin/env python3

ts = time.time update_interval = 120
 * 1) For timing, in seconds

headers = {'Connection': 'Keep alive', 'Content-Type': 'application/x-www-form-urlencoded', 'User-Agent': 'Atvelonis/Bot'}
 * 1) Info for the web server. https://en.wikipedia.org/wiki/HTTP_persistent_connection
 * 2) The API for Discussions asks for Content-Type, User-Agent

wiki = 'elderscrolls' username = 'AkulakhanBot' password = ''
 * 1) Account credentials for the bot to log in

session = core.login(wiki, username, password) print(core.is_logged_in(session, username, wiki)) # Boolean: check if logged in or not wiki_id = core.get_wiki_id(session, wiki) print(wiki_id)
 * 1) Calls on functions in core to log in and begin edit session

payload = {'limit': '25', 'page': '0', 'responseGroup': 'small', 'reported': 'false', 'viewableOnly': 'true'} r = session.get('https://services.fandom.com/discussion/'+wiki_id+'/posts', params=payload, headers={'Accept': 'application/hal+json', 'User-Agent': 'Atvelonis/Bot'}) print(r) # Should be 200
 * 1) Vague filter terms that Wikia defined

for post in reversed(r.json['_embedded']['doc:posts']): if int(post['creationDate']['epochSecond']) > ts - update_interval: content = post['rawContent'] name = post['createdBy']['name'] avatar = post['createdBy']['avatarUrl'] forum_name = post['forumName'] thread_id = post['threadId'] user_id = post['createdBy']['id'] post_id = post['id'] thread_title = post['_embedded']['thread'][0]['title'] post_epoch = post['creationDate']['epochSecond'] if post['isReply']: thread_title = post['_embedded']['thread'][0]['title'] + ' (reply)' is_reply = post['isReply'] print(name + ' says: ' + content) # Content the bot deletes if 'Like, C0DA makes it canon, dude.' in content: print('Disallowed content matched') session.put('https://services.fandom.com/discussion/'+wiki_id+'/posts/'+post_id+'/delete', headers={'Accept': 'application/hal+json', 'User-Agent': 'Atvelonis/Bot'}) print('Deleted: https://elderscrolls.fandom.com/d/p/'+thread_id+'/r/'+post_id+' and content was: '+content)
 * 1) Views posts made in the last two minutes and deletes them if needed


 * Alternative for deletion code

blacklist = ['foo', 'bar'] if any(i in content for i in blacklist): print('Disallowed content matched') session.put('https://services.fandom.com/discussion/'+wiki_id+'/posts/'+post_id+'/delete', headers={'Accept': 'application/hal+json', 'User-Agent': 'Atvelonis/Bot'}) print('Deleted: https://elderscrolls.fandom.com/d/p/'+thread_id+'/r/'+post_id+' and content was: '+content)


 * Regex deletion example

if re.match(r'Click Here https:\/\/(.\.)*.+\..+\/.+-*', content): {{Clr}}

{{User:Atvelonis/AtvelonisSubpageNav}}