BioPathBot

Description

Depuis une databiographie (correction(s): ) suivant le format utilisé sur le wiki (correction(s): kiki, tiki ) (ex (correction(s): exo, en, eu, eux, es, eh, et, x, e, rex, tex ): 1890.03.19 / Genève), le BioPathBot est capable d'extraire les informations temporelles et spatiales. Ces deux données forment un tuple (correction(s): tuile, tulle ) et sont parsées (correction(s): parées, pansées, parlées, passées ) de façon suivante :

• La date est formée d'une année, d'un mois et d'un jour. Si des informations venaient à manquer, la date est complétée par défaut avec le mois de janvier et le premier du mois.
• Le lieu est transformé en coordonnées géographiques (utilisant la librairie python [https (correction(s):

)://pypi.python.org (correction(s): orge, ore, ors, or, ort, ord, erg )/pypi/geopy (correction(s): ) geopy (correction(s): )])

Ainsi transformées et triées chronologiquement, ces données permettent de générer une carte utilisant [http (correction(s): )://matplotlib (correction(s): ).org (correction(s): orge, ore, ors, or, ort, ord, erg )/basemap (correction(s): )/index.html basemap (correction(s): )], où sont dessinés des points pour marquer les lieux, ainsi que des traits mettant en avant la trajectoire de la personne. Cette carte est ensuite importée sur wikipast (correction(s): ) puis insérée sur une page annexe.

Par exemple la carte de Jean Tinguely se visualise comme:

Critique

• Peu importe le type de page (une page concernant une personne, un hypermot...), une carte est générée. Donc si une page ne relate pas la vie d'une personne, la carte aura peu de sens.

• Avec l'algorithme actuel, les dates et les lieux sont parfaitement extraits et placé sur la carte. Cependant, si les données ne sont pas suffisamment précises, elles seront approximées (correction(s):

) (1980 devient 1980.01.01 tandis (correction(s): candis, tendis, taudis ) que le point géographique correspondant à la Suisse sera son centre géographique). Cela peut mener à des erreurs de trajectoire lors du tri chronologique ou simplement à une position différente de la réalité sur la carte.

• S'il existe des événements post-mortem (correction(s): mortel, mortes, morte

) mais que la mort de la personne n'est pas mentionnée ou ne suit pas la convention de l'hypermot Décès, ces données seront aussi extraites et placées sur la carte, alors même que la personne ne se déplace pas.

• Si des événements sont spatialement trop proches, ils vont se superposer sur la carte et il deviendra difficile de retracer les événements d'une façon visuellement claire. Par exemple, pour un trait entre Berne et Lausanne, il sera difficile de juger si une personne a fait plusieurs fois ce même déplacement ou ne l'a fait qu'une seule fois au cours de sa vie.

• Parfois l'outil qui transforme les lieux en coordonnées géographiques crée des erreurs. Par exemple "Rome" devient "Lomé" qui se situe au Togo.

• Certains événements se passe sans que la personne soit physiquement présente, mais on ne peut pas le détecter automatiquement. Les lieux apparaissent donc sur la carte.

Code

 import urllib (correction(s): 

).request (correction(s): )

 import requests (correction(s): 

)

 from (correction(s): rom, froc

) bs (correction(s): bé, ès, s, by, bi, bis, b, es, bus, bas, as, bu, os, us )4 import BeautifulSoup

 import re
 import math
 import datetime (correction(s): 

)

 from (correction(s): rom, froc

) geopy (correction(s): ).geocoders (correction(s): ) import Nominatim

 from (correction(s): rom, froc

) mpl (correction(s): mil, mal, mol, ml )_toolkits (correction(s): ).basemap (correction(s): ) import Basemap

 import matplotlib (correction(s): 

).pyplot (correction(s): ) as plt (correction(s): plut, plet, pet, pût, plût, pli, plot, plat, pst, put, pât, plu, pat, pot )

 from (correction(s): rom, froc

) colorsys (correction(s): ) import hsv (correction(s): hiv )_to_rgb (correction(s): rab )

 from (correction(s): rom, froc

) matplotlib (correction(s): ).colors (correction(s): colore, coloris, colorés, colos, colora, coloré, colons ) import rgb (correction(s): rab )2hex (correction(s): hem, heu, hep, rex, tex )

 SEGMENTS = 100
 
 # draw (correction(s): drap, drag

) plots inline (correction(s): incline ) rather (correction(s): ratier, rater ) than (correction(s): tian, ahan, tan, thon, thaï, han, khan ) in a seperate (correction(s): ) window (correction(s): )

 %matplotlib (correction(s): 

) inline (correction(s): incline )

 # draw (correction(s): drap, drag

) plots bigger (correction(s): bigler, jigger )

 plt (correction(s): plut, plet, pet, pût, plût, pli, plot, plat, pst, put, pât, plu, pat, pot

).rcParams["figure.figsize"] = [20.0, 10.0]

 user='BioPathBot'
 passw (correction(s): passe, passé, passa

)='chkiroju (correction(s): )'

 baseurl (correction(s): 

)='http (correction(s): )://wikipast (correction(s): ).epfl (correction(s): ).ch (correction(s): chu, co, h, ce, ah, eh, c, ph, oh, ci, cm, chi )/wikipast (correction(s): )/'

 summary (correction(s): 

)='Wikipastbot update'

 protected (correction(s): 

)_logins (correction(s): logis, login, lopins, logions )=["Frederickaplan","Maud","Vbuntinx","Testbot","IB","SourceBot","PageUpdaterBot","Orthobot","BioPathBot","ChronoBOT","Amonbaro","AntoineL","AntoniasBan deros","Arnau","Arnaudpannatier","Aureliver","Brunowicht","Burgerpop","Cedricviaccoz","Christophe","Claudioloureiro","Ghislain","Gregoire3245","Hirtg","Houssm","Icebak er","JenniCin","JiggyQ","JulienB","Kl","Kperrard","Leandro Kieliger","Marcus","Martin","MatteoGiorla","Mireille","Mj2905","Musluoglucem","Nacho","Nameless","Nawel","O'showa","PA","Qantik","QuentinB","Raphael.barman","Roblan11" ,"Romain Fournier","Sbaaa","Snus","Sonia","Tboyer","Thierry","Titi","Vlaedr","Wanda"]

 depuis_date='2017-05-02T16:00:00Z'
 liste_pages=[]
 for user in protected (correction(s): 

)_logins (correction(s): logis, login, lopins, logions ):

     result (correction(s): 

)=requests (correction(s): ).post(baseurl (correction(s): )+'api.php (correction(s): phi, pep, pop, ph )?action=query (correction(s): )&list (correction(s): liste, listé, lise, lest, lis, liât, lift, lit )=usercontribs (correction(s): )&ucuser (correction(s): )='+user+'&format=xml (correction(s): ml )&ucend (correction(s): )='+depuis_date)

     soup (correction(s): houp, stup, soupe, soupé, youp, soul, sous, souk, soupa, sou, coup, loup, soue

)=BeautifulSoup(result (correction(s): ).content,'lxml (correction(s): )')

     for primitive in soup (correction(s): houp, stup, soupe, soupé, youp, soul, sous, souk, soupa, sou, coup, loup, soue

).usercontribs (correction(s): ).findAll('item'):

         liste_pages.append (correction(s): appends, appendu, apprend

)(primitive['title (correction(s): tille, titre )'])

         print (correction(s): peint, prin, priant, priât, point, sprint, prit, prient

)(primitive['title (correction(s): tille, titre )'])

 names (correction(s): nases, dames, lames, nages, nazes, cames, rames

)=list (correction(s): liste, listé, lise, lest, lis, liât, lift, lit )(set(liste_pages))

 # Login request (correction(s): 

)

 payload (correction(s): 

)={'action':'query (correction(s): )','format':'json (correction(s): son )','utf (correction(s): urf, ut )8':,'meta (correction(s): jeta, feta, mata, méta, mena, mets, met, mesa, mita )':'tokens (correction(s): )','type':'login'}

 r (correction(s): ô, , k, re, ri, q, or, z, v, t, l, é, n, b, o, j, m, ru, s, d, y, a, f, e, à, ré, h, c, x, g, w, u, ra, i

)1=requests (correction(s): ).post(baseurl (correction(s): ) + 'api.php (correction(s): phi, pep, pop, ph )', data=payload (correction(s): ))

 #login confirm (correction(s): confire, confirma, confirme, confirmé

)

 login_token (correction(s): 

)=r1.json (correction(s): son )()['query (correction(s): )']['tokens (correction(s): )']['logintoken']

 payload (correction(s): 

)={'action':'login','format':'json (correction(s): son )','utf (correction(s): urf, ut )8':,'lgname (correction(s): igname )':user,'lgpassword (correction(s): )':passw,'lgtoken (correction(s): )':login_token}

 r (correction(s): ô, , k, re, ri, q, or, z, v, t, l, é, n, b, o, j, m, ru, s, d, y, a, f, e, à, ré, h, c, x, g, w, u, ra, i

)2=requests (correction(s): ).post(baseurl (correction(s): ) + 'api.php (correction(s): phi, pep, pop, ph )', data=payload (correction(s): ), cookies=r (correction(s): ô, , k, re, ri, q, or, z, v, t, l, é, n, b, o, j, m, ru, s, d, y, a, f, e, à, ré, h, c, x, g, w, u, ra, i )1.cookies)

 #get (correction(s): cet, pet, let, gel, guet, ket, gît, net, et, set, met, jet, gent, gut

) edit (correction(s): redit, dit, exit, édit, ledit ) token (correction(s): )2

 params (correction(s): parais, paras

)3='?format=json (correction(s): son )&action=query (correction(s): )&meta (correction(s): jeta, feta, mata, méta, mena, mets, met, mesa, mita )=tokens (correction(s): )&continue='

 r (correction(s): ô, , k, re, ri, q, or, z, v, t, l, é, n, b, o, j, m, ru, s, d, y, a, f, e, à, ré, h, c, x, g, w, u, ra, i

)3=requests (correction(s): ).get (correction(s): cet, pet, let, gel, guet, ket, gît, net, et, set, met, jet, gent, gut )(baseurl (correction(s): ) + 'api.php (correction(s): phi, pep, pop, ph )' + params (correction(s): parais, paras )3, cookies=r2.cookies)

 edit (correction(s): redit, dit, exit, édit, ledit

)_token (correction(s): )=r3.json (correction(s): son )()['query (correction(s): )']['tokens (correction(s): )']['csrftoken']

 edit (correction(s): redit, dit, exit, édit, ledit

)_cookie=r (correction(s): ô, , k, re, ri, q, or, z, v, t, l, é, n, b, o, j, m, ru, s, d, y, a, f, e, à, ré, h, c, x, g, w, u, ra, i )2.cookies.copy (correction(s): cops, cosy, cop )()

 edit (correction(s): redit, dit, exit, édit, ledit

)_cookie.update(r (correction(s): ô, , k, re, ri, q, or, z, v, t, l, é, n, b, o, j, m, ru, s, d, y, a, f, e, à, ré, h, c, x, g, w, u, ra, i )3.cookies)

 #setup (correction(s): stup

) geolocator (correction(s): )

 geolocator (correction(s): 

) = Nominatim(timeout (correction(s): )=10)

 # upload (correction(s): 

) config (correction(s): confit, confié, confia, confie )

 def (correction(s): zef, der, des, nef, de, dey

) uploadMap(filename (correction(s): )):

     # read (correction(s): rend, rad

) local file

     upload (correction(s): 

)_file = open(filename (correction(s): ),"rb (correction(s): ru, ré, b, rab, re, ri, ra, ob )")

     upload (correction(s): 

)_contents = upload (correction(s): )_file.read (correction(s): rend, rad )()

     upload (correction(s): 

)_file.close()

     # setting (correction(s): betting

) parameters (correction(s): ) for upload (correction(s): )

     # ref (correction(s): zef, rez, rep, re, rif, nef, reg, bref, rex, réf

): https (correction(s): )://www (correction(s): ).mediawiki (correction(s): ).org (correction(s): orge, ore, ors, or, ort, ord, erg )/wiki (correction(s): kiki, tiki )/API:Upload

     payload (correction(s): 

)={'action':'upload (correction(s): )','filename (correction(s): )':filename (correction(s): ), 'ignorewarnings (correction(s): )':1, 'token (correction(s): )':edit (correction(s): redit, dit, exit, édit, ledit )_token (correction(s): )}

     files={'file':upload (correction(s): 

)_contents}

     # upload (correction(s): 

) the (correction(s): tee, te, tue, thé ) image

     print (correction(s): peint, prin, priant, priât, point, sprint, prit, prient

)("Uploading file to %s via API..." % (baseurl (correction(s): )+"index.php (correction(s): phi, pep, pop, ph )/Fichier:"+filename (correction(s): )))

     r4=requests (correction(s): 

).post(baseurl (correction(s): )+'api.php (correction(s): phi, pep, pop, ph )',data=payload (correction(s): ),files=files,cookies=edit (correction(s): redit, dit, exit, édit, ledit )_cookie)

     # in case of (correction(s): bof, f, on, if, ou, lof, o, off, oh, où, or, ob, ok, ouf, os

) error (correction(s): errer ) print (correction(s): peint, prin, priant, priât, point, sprint, prit, prient ) the (correction(s): tee, te, tue, thé ) response (correction(s): )

     #print (correction(s): peint, prin, priant, priât, point, sprint, prit, prient

)(r4.text (correction(s): texto, texte, test, tex ))

 def (correction(s): zef, der, des, nef, de, dey

) addToPage(name (correction(s): nave, naze, rame, mame, lame, dame, game, came, nase, nage, nome ), img (correction(s): )):

     title (correction(s): tille, titre

) = name (correction(s): nave, naze, rame, mame, lame, dame, game, came, nase, nage, nome ) + " BioPathBot"

     content = "[[Fichier: "+ img (correction(s): 

) +"]]"

     pageToChange = requests (correction(s): 

).post(baseurl (correction(s): )+'api.php (correction(s): phi, pep, pop, ph )?action=query (correction(s): )&titles (correction(s): tilles, titres )='+title (correction(s): tille, titre )+'&export&exportnowrap (correction(s): )')

     payload (correction(s): 

)={'action':'edit (correction(s): redit, dit, exit, édit, ledit )','assert (correction(s): )':'user','format':'json (correction(s): son )','utf (correction(s): urf, ut )8':,'text (correction(s): texto, texte, test, tex )':content,'summary (correction(s): )':summary (correction(s): ),'title (correction(s): tille, titre )':title (correction(s): tille, titre ),'token (correction(s): )':edit (correction(s): redit, dit, exit, édit, ledit )_token (correction(s): )}

     r (correction(s): ô, , k, re, ri, q, or, z, v, t, l, é, n, b, o, j, m, ru, s, d, y, a, f, e, à, ré, h, c, x, g, w, u, ra, i

)4=requests (correction(s): ).post(baseurl (correction(s): )+'api.php (correction(s): phi, pep, pop, ph )',data=payload (correction(s): ),cookies=edit (correction(s): redit, dit, exit, édit, ledit )_cookie)

 # BioPathBot : add (correction(s): ado, aïd, adp, ada

) line of (correction(s): bof, f, on, if, ou, lof, o, off, oh, où, or, ob, ok, ouf, os ) databiographie (correction(s): ) to the (correction(s): tee, te, tue, thé ) right (correction(s): light ) page (time (correction(s): tire, mime, tige, cime, tome, tipe, lime, aime, trime, rime, tine ) and (correction(s): ans, aïd, land, an, anda, ana ) space (correction(s): espace ))

 def (correction(s): zef, der, des, nef, de, dey

) getDataFromPage(name (correction(s): nave, naze, rame, mame, lame, dame, game, came, nase, nage, nome )):

     data = []
     result (correction(s): 

)=requests (correction(s): ).post(baseurl (correction(s): )+'api.php (correction(s): phi, pep, pop, ph )?action=query (correction(s): )&titles (correction(s): tilles, titres )='+name (correction(s): nave, naze, rame, mame, lame, dame, game, came, nase, nage, nome )+'&export&exportnowrap (correction(s): )')

     soup (correction(s): houp, stup, soupe, soupé, youp, soul, sous, souk, soupa, sou, coup, loup, soue

)=BeautifulSoup(result (correction(s): ).text (correction(s): texto, texte, test, tex ), "lxml (correction(s): )")

     #soup (correction(s): houp, stup, soupe, soupé, youp, soul, sous, souk, soupa, sou, coup, loup, soue

)=BeautifulSoup(result (correction(s): ).text (correction(s): texto, texte, test, tex ))

     code=
     for primitive in soup (correction(s): houp, stup, soupe, soupé, youp, soul, sous, souk, soupa, sou, coup, loup, soue

).findAll("text (correction(s): texto, texte, test, tex )"):

         code+=primitive.string
 
     # split (correction(s): salit

) on list (correction(s): liste, listé, lise, lest, lis, liât, lift, lit ) (*)

     lines (correction(s): vines, liées, linos, lignes, mines, lises, lins, limes, lires, lices, lianes, laines, linges, lunes, liges, lies, pines, aines, line, fines

) = code.split (correction(s): salit )("*")

     for line in lines (correction(s): vines, liées, linos, lignes, mines, lises, lins, limes, lires, lices, lianes, laines, linges, lunes, liges, lies, pines, aines, line, fines

) :

         # add (correction(s): ado, aïd, adp, ada

) breaking (correction(s): ) lines (correction(s): vines, liées, linos, lignes, mines, lises, lins, limes, lires, lices, lianes, laines, linges, lunes, liges, lies, pines, aines, line, fines ) (otherwise (correction(s): ) will (correction(s): till, bill, twill ) be (correction(s): bé, bi, re, ne, bée, de, beu, bye, by, b, bue, se, le, bel, bey, e, bu, me, te, ce, je, bec, ber, ben ) appened (correction(s): ) directly (correction(s): ) in one (correction(s): ont, gone, oie, ose, onde, on, ope, none, onc, ode, oye, zone, orne, ore, ne, âne, onze, once, ove, une ) line)

         line = "\n\n"+line
 
         # get (correction(s): cet, pet, let, gel, guet, ket, gît, net, et, set, met, jet, gent, gut

) date if exist (correction(s): exista, existé, existe, exit )

         date = re.findall (correction(s): 

)("((?<=\[\[)\d*(\.*\d*\.*\d*)*(?=\]\]))",line)

         dateToAdd = ""
 
         if len(date) != 0 :
             dateToAdd = date[0][0]
 
         # get (correction(s): cet, pet, let, gel, guet, ket, gît, net, et, set, met, jet, gent, gut

) place if exist (correction(s): exista, existé, existe, exit )

         place = re.findall (correction(s): 

)("(?<=\/\s\[\[)[A-zÀ-ÿ]*(?=\]\])",line)

         location = ""
         if len(place) != 0:
              placeToAdd = place[0]
              location = geolocator (correction(s): 

).geocode(placeToAdd)

         # if both the (correction(s): tee, te, tue, thé

) date and (correction(s): ans, aïd, land, an, anda, ana ) the (correction(s): tee, te, tue, thé ) location are available, append (correction(s): appends, appendu, apprend ) in data array

         if dateToAdd and (correction(s): ans, aïd, land, an, anda, ana

) location:

             dataToAdd = [location.longitude,location.latitude];
             data.append (correction(s): appends, appendu, apprend

)(dataToAdd);

     return data
 
 
 # finds the (correction(s): tee, te, tue, thé

) minimal and (correction(s): ans, aïd, land, an, anda, ana ) maximal longitude and (correction(s): ans, aïd, land, an, anda, ana ) latitude

 def (correction(s): zef, der, des, nef, de, dey

) findCorners(pts):

     minlon = maxlon = pts[0][0]
     minlat = maxlat = pts[0][1]
     for p in pts:
         currlon = p[0]
         if currlon<minlon:
             minlon = currlon
         elif currlon>maxlon:
             maxlon = currlon
 
         currlat = p[1]
         if currlat<minlat:
             minlat = currlat
         elif currlat>maxlat:
             maxlat = currlat
 
     return [minlon, maxlon, minlat, maxlat]
 
 # draws the (correction(s): tee, te, tue, thé

) map, some points and (correction(s): ans, aïd, land, an, anda, ana ) the (correction(s): tee, te, tue, thé ) lines (correction(s): vines, liées, linos, lignes, mines, lises, lins, limes, lires, lices, lianes, laines, linges, lunes, liges, lies, pines, aines, line, fines )

 def (correction(s): zef, der, des, nef, de, dey

) drawmap(pts, filename (correction(s): ), export=False):

     n_pts = len(pts)
     corners = findCorners(pts)
     m = Basemap(llcrnrlon=corners[0]-1, llcrnrlat=corners[2]-1, urcrnrlon=corners[1]+1, urcrnrlat=corners[3]+1, resolution='i')
     m.drawmapboundary(fill_color='0.6')
     m.drawcountries(linewidth=1.0, color='0.6')
     m.fillcontinents(color='white', lake_color='white')
     for i in range(n_pts-1): # draw (correction(s): drap, drag

) lines (correction(s): vines, liées, linos, lignes, mines, lises, lins, limes, lires, lices, lianes, laines, linges, lunes, liges, lies, pines, aines, line, fines )

         for j in range(SEGMENTS):
             start = pts[i] + (pts[i+1]-pts[i])*(j/SEGMENTS)
             end = pts[i] + (pts[i+1]-pts[i])*((j+1)/SEGMENTS)
             m.plot([start[0], end[0]], [start[1], end[1]], color=hsv (correction(s): hiv

)_to_rgb (correction(s): rab )((i+j/SEGMENTS)/n_pts, 1, 1))

     for i in range(n_pts): # draw (correction(s): drap, drag

) points

         m.plot(pts[i][0], pts[i][1], marker='o', color=hsv (correction(s): hiv

)_to_rgb (correction(s): rab )(i/n_pts, 1, 1), fillstyle='full', markeredgewidth=0.0)

     if export:
         plt (correction(s): plut, plet, pet, pût, plût, pli, plot, plat, pst, put, pât, plu, pat, pot

).savefig(filename (correction(s): ), bbox_inches='tight')

     plt (correction(s): plut, plet, pet, pût, plût, pli, plot, plat, pst, put, pât, plu, pat, pot

).show()

 for name (correction(s): nave, naze, rame, mame, lame, dame, game, came, nase, nage, nome

) in names (correction(s): nases, dames, lames, nages, nazes, cames, rames ):

     image_filename (correction(s): 

) = (name (correction(s): nave, naze, rame, mame, lame, dame, game, came, nase, nage, nome ) + "_biopath.png").replace(" ","_")

     data = getDataFromPage(name (correction(s): nave, naze, rame, mame, lame, dame, game, came, nase, nage, nome

))

     drawmap(np.array(data), image_filename (correction(s): 

), True)

     uploadMap(image_filename (correction(s): 

))

     addToPage(name, image_filename (correction(s): 

))

Groupe

Nom et Prénom	Pseudo
Christophe Badoux	Christophe
Julien Burkhard	JulienB
Kim Lan Phan Hoang	Kl
Robin Lang	Roblan11