ben
/
copyvios
peilaus alkaen https://github.com/earwig/copyvios
1
1
Fork 0
Koodi Julkaisut 0 Toiminta
A copyright violation detector running on Wikimedia Cloud Services https://tools.wmflabs.org/copyvios/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
442 Commitit
4 Branchit
1.4 MiB
 
 
 
 
 
Puu: c9066e6259
Branchit Tagit
${ item.name }
Create branch ${ searchTerm }
from 'c9066e6259'
${ noResults }
copyvios/templates/index.mako

324 rivejä
21 KiB
Raaka Blame Historia 

  1. <%!

  2.     from flask import request

  3.     from copyvios.attribution import get_attribution_info

  4.     from copyvios.checker import T_POSSIBLE, T_SUSPECT

  5.     from copyvios.cookies import get_cookies

  6.     from copyvios.misc import cache

  7. %>\

  8. <%

  9.     titleparts = []

  10.     if query.page:

  11.         titleparts.append(query.page.title)

  12.     titleparts.append("Earwig's Copyvio Detector")

  13.     title = " | ".join(titleparts)

  14.     cookies = get_cookies()

  15. %>\

  16. <%include file="/support/header.mako" args="title=title, splash=not result"/>

  17. <%namespace module="copyvios.highlighter" import="highlight_delta"/>\

  18. <%namespace module="copyvios.misc" import="httpsfix, urlstrip"/>\

  19. % if notice:

  20.     <div id="notice-box" class="gray-box">

  21.         ${notice}

  22.     </div>

  23. % endif

  24. % if query.submitted:

  25.     % if query.error:

  26.         <div id="info-box" class="red-box"><p>

  27.             % if query.error == "bad action":

  28.                 Unknown action: <b><span class="mono">${query.action | h}</span></b>.

  29.             % elif query.error == "no search method":

  30.                 No copyvio search methods were selected. A check can only be made using the search engine, links present in the page, Turnitin, or some combination of these.

  31.             % elif query.error == "bad oldid":

  32.                 The revision ID <code>${query.oldid | h}</code> is invalid. It should be an integer.

  33.             % elif query.error == "no URL":

  34.                 Compare mode requires a URL to be entered. Enter one in the text box below, or choose copyvio search mode to look for content similar to the article elsewhere on the web.

  35.             % elif query.error == "bad URI":

  36.                 Unsupported URI scheme: <a href="${query.url | h}">${query.url | h}</a>.

  37.             % elif query.error == "no data":

  38.                 Couldn't find any text in <a href="${query.url | h}">${query.url | h}</a>. <i>Note:</i> only HTML documents, plain text pages, and PDFs are supported, and content generated by JavaScript or found inside iframes is ignored.

  39.             % elif query.error == "timeout":

  40.                 The URL <a href="${query.url | h}">${query.url | h}</a> timed out before any data could be retrieved.

  41.             % elif query.error == "search error":

  42.                 An error occurred while using the search engine (${query.error.__cause__}). <i>Note:</i> there is a daily limit on the number of search queries the tool is allowed to make. You may <a href="${request.url | httpsfix, h}&amp;use_engine=0">repeat the check without using the search engine</a>.

  43.             % else:

  44.                 An unknown error occurred.

  45.             % endif

  46.         </p></div>

  47.     % elif not query.site:

  48.         <div id="info-box" class="red-box">

  49.             <p>The given site (project=<b><span class="mono">${query.project | h}</span></b>, language=<b><span class="mono">${query.lang | h}</span></b>) doesn't seem to exist. It may also be closed or private. <a href="https://${query.lang | h}.${query.project | h}.org/">Confirm its URL.</a></p>

  50.         </div>

  51.     % elif query.oldid and not result:

  52.         <div id="info-box" class="red-box">

  53.             <p>The revision ID couldn't be found: <a href="https://${query.site.domain | h}/w/index.php?oldid=${query.oldid | h}">${query.oldid | h}</a>.</p>

  54.         </div>

  55.     % elif query.title and not result:

  56.         <div id="info-box" class="red-box">

  57.             <p>The page couldn't be found: <a href="${query.page.url}">${query.page.title | h}</a>.</p>

  58.         </div>

  59.     % endif

  60. %endif

  61. <p>This tool attempts to detect <a href="https://en.wikipedia.org/wiki/WP:COPYVIO">copyright violations</a> in articles. In <i>search mode</i>, it will check for similar content elsewhere on the web using <a href="https://developers.google.com/custom-search/">Google</a>, external links present in the text of the page, or <a href="https://en.wikipedia.org/wiki/Wikipedia:Turnitin">Turnitin</a> (via <a href="https://en.wikipedia.org/wiki/User:EranBot">EranBot</a>), depending on which options are selected. In <i>compare mode</i>, the tool will compare the article to a specific webpage without making additional searches, like the <a href="https://dupdet.toolforge.org/">Duplication Detector</a>.</p>

  62. <p>Running a full check can take up to a minute if other websites are slow or if the tool is under heavy use. Please be patient. If you get a timeout, wait a moment and refresh the page.</p>

  63. <p>Be aware that other websites can copy from Wikipedia, so check the results carefully, especially for older or well-developed articles. Specific websites can be skipped by adding them to the <a href="https://en.wikipedia.org/wiki/User:EarwigBot/Copyvios/Exclusions">excluded URL list</a>.</p>

  64. <form id="cv-form" action="${request.script_root}/" method="get">

  65.     <div class="oo-ui-layout oo-ui-horizontalLayout">

  66.         <label class="site oo-ui-widget oo-ui-widget-enabled oo-ui-labelElement-label oo-ui-labelElement oo-ui-labelWidget">Site</label>

  67.         <div class="oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-dropdownInputWidget oo-ui-dropdownInputWidget-php">

  68.             <select name="lang" required="" class="oo-ui-inputWidget-input oo-ui-indicator-down" title="Language">

  69.                 <% selected_lang = query.orig_lang if query.orig_lang else cookies["CopyviosDefaultLang"].value if "CopyviosDefaultLang" in cookies else cache.bot.wiki.get_site().lang %>\

  70.                 % for code, name in cache.langs:

  71.                     % if code == selected_lang:

  72.                         <option value="${code | h}" selected="selected">${name}</option>

  73.                     % else:

  74.                         <option value="${code | h}">${name}</option>

  75.                     % endif

  76.                 % endfor

  77.             </select>

  78.         </div>

  79.         <div class="oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-dropdownInputWidget oo-ui-dropdownInputWidget-php">

  80.             <select name="project" required="" class="oo-ui-inputWidget-input oo-ui-indicator-down" title="Project">

  81.                 <% selected_project = query.project if query.project else cookies["CopyviosDefaultProject"].value if "CopyviosDefaultProject" in cookies else cache.bot.wiki.get_site().project %>\

  82.                 % for code, name in cache.projects:

  83.                     % if code == selected_project:

  84.                         <option value="${code | h}" selected="selected">${name}</option>

  85.                     % else:

  86.                         <option value="${code | h}">${name}</option>

  87.                     % endif

  88.                 % endfor

  89.             </select>

  90.         </div>

  91.     </div>

  92.     <div class="oo-ui-layout oo-ui-horizontalLayout">

  93.         <label for="cv-title" class="page oo-ui-widget oo-ui-widget-enabled oo-ui-labelElement-label oo-ui-labelElement oo-ui-labelWidget">Page</label>

  94.         <div class="page-title oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-textInputWidget oo-ui-textInputWidget-type-text oo-ui-textInputWidget-php">

  95.             <input id="cv-title" type="text" class="oo-ui-inputWidget-input" name="title" placeholder="Title" title="Page title"

  96.                 % if query.title:

  97.                     value="${query.page.title if query.page else query.title | h}"

  98.                 % endif

  99.             >

  100.         </div>

  101.         <label class="oo-ui-widget oo-ui-widget-enabled oo-ui-labelElement-label oo-ui-labelElement oo-ui-labelWidget">or</label>

  102.         <div class="page-oldid oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-textInputWidget oo-ui-textInputWidget-type-text oo-ui-textInputWidget-php">

  103.             <input id="cv-oldid" type="text" class="oo-ui-inputWidget-input" name="oldid" placeholder="Revision ID" title="Revision ID"

  104.                 % if query.oldid:

  105.                     value="${query.oldid | h}"

  106.                 % endif

  107.             >

  108.         </div>

  109.     </div>

  110.     <div class="oo-ui-layout oo-ui-horizontalLayout">

  111.         <span class="oo-ui-widget oo-ui-widget-enabled">

  112.             <span class="oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-radioInputWidget">

  113.                 <input id="action-search" class="oo-ui-inputWidget-input" type="radio" name="action" value="search" ${'checked="checked"' if (query.action == "search" or not query.action) else ""}><span></span>

  114.             </span>

  115.             <label for="action-search" class="action oo-ui-widget oo-ui-widget-enabled oo-ui-labelElement-label oo-ui-labelElement oo-ui-labelWidget">Copyvio search</label>

  116.         </span>

  117. 

  118.         <input type="hidden" name="use_engine" value="0">

  119.         <span class="oo-ui-widget oo-ui-widget-enabled">

  120.             <span class="cv-search-oo-ui oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-checkboxInputWidget">

  121.                 <input id="cv-cb-engine" class="cv-search oo-ui-inputWidget-input" type="checkbox" name="use_engine" value="1" ${'checked="checked"' if query.use_engine not in ("0", "false") else ""}>

  122.                 <span class="oo-ui-checkboxInputWidget-checkIcon oo-ui-widget oo-ui-widget-enabled oo-ui-iconElement-icon oo-ui-icon-check oo-ui-iconElement oo-ui-labelElement-invisible oo-ui-iconWidget oo-ui-image-invert"></span>

  123.             </span>

  124.             <label for="cv-cb-engine" class="oo-ui-widget oo-ui-widget-enabled oo-ui-labelElement-label oo-ui-labelElement oo-ui-labelWidget">Use search engine</label>

  125.         </span>

  126. 

  127.         <input type="hidden" name="use_links" value="0">

  128.         <span class="oo-ui-widget oo-ui-widget-enabled">

  129.             <span class="cv-search-oo-ui oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-checkboxInputWidget">

  130.                 <input id="cv-cb-links" class="cv-search oo-ui-inputWidget-input" type="checkbox" name="use_links" value="1" ${'checked="checked"' if query.use_links not in ("0", "false") else ""}>

  131.                 <span class="oo-ui-checkboxInputWidget-checkIcon oo-ui-widget oo-ui-widget-enabled oo-ui-iconElement-icon oo-ui-icon-check oo-ui-iconElement oo-ui-labelElement-invisible oo-ui-iconWidget oo-ui-image-invert"></span>

  132.             </span>

  133.             <label for="cv-cb-links" class="oo-ui-widget oo-ui-widget-enabled oo-ui-labelElement-label oo-ui-labelElement oo-ui-labelWidget">Use links in page</label>

  134.         </span>

  135. 

  136.         <input type="hidden" name="turnitin" value="0">

  137.         <span class="oo-ui-widget oo-ui-widget-enabled">

  138.             <span class="cv-search-oo-ui oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-checkboxInputWidget">

  139.                 <input id="cv-cb-turnitin" class="cv-search oo-ui-inputWidget-input" type="checkbox" name="turnitin" value="1" ${'checked="checked"' if query.turnitin in ("1", "true") else ""}>

  140.                 <span class="oo-ui-checkboxInputWidget-checkIcon oo-ui-widget oo-ui-widget-enabled oo-ui-iconElement-icon oo-ui-icon-check oo-ui-iconElement oo-ui-labelElement-invisible oo-ui-iconWidget oo-ui-image-invert"></span>

  141.             </span>

  142.             <label for="cv-cb-turnitin" class="oo-ui-widget oo-ui-widget-enabled oo-ui-labelElement-label oo-ui-labelElement oo-ui-labelWidget">Use Turnitin</label>

  143.         </span>

  144.     </div>

  145.     <div class="oo-ui-layout oo-ui-horizontalLayout">

  146.         <span class="oo-ui-widget oo-ui-widget-enabled">

  147.             <span class="oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-radioInputWidget">

  148.                 <input id="action-compare" class="oo-ui-inputWidget-input" type="radio" name="action" value="compare" ${'checked="checked"' if query.action == "compare" else ""}><span></span>

  149.             </span>

  150.             <label for="action-compare" class="action oo-ui-widget oo-ui-widget-enabled oo-ui-labelElement-label oo-ui-labelElement oo-ui-labelWidget">Copyvio compare</label>

  151.         </span>

  152.         <div class="compare-url cv-compare-oo-ui oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-textInputWidget oo-ui-textInputWidget-type-text oo-ui-textInputWidget-php">

  153.             <input type="text" class="cv-compare oo-ui-inputWidget-input" name="url" placeholder="URL" title="URL to compare"

  154.                 % if query.url:

  155.                     value="${query.url | h}"

  156.                 % endif

  157.             >

  158.         </div>

  159.     </div>

  160.     % if query.nocache or (result and result.cached):

  161.         <div class="oo-ui-layout oo-ui-horizontalLayout">

  162.             <span class="cv-search-oo-ui oo-ui-widget oo-ui-widget-enabled oo-ui-inputWidget oo-ui-checkboxInputWidget">

  163.                 <input id="cb-nocache" class="oo-ui-inputWidget-input" type="checkbox" name="nocache" value="1" ${'checked="checked"' if query.nocache else ""}>

  164.                 <span class="oo-ui-checkboxInputWidget-checkIcon oo-ui-widget oo-ui-widget-enabled oo-ui-iconElement-icon oo-ui-icon-check oo-ui-iconElement oo-ui-labelElement-invisible oo-ui-iconWidget oo-ui-image-invert"></span>

  165.             </span>

  166.             <label for="cb-nocache">Bypass cache</label>

  167.         </div>

  168.     % endif

  169.     <div class="oo-ui-layout oo-ui-horizontalLayout">

  170.         <span class="oo-ui-widget oo-ui-widget-enabled oo-ui-buttonElement oo-ui-buttonElement-framed oo-ui-labelElement oo-ui-flaggedElement-primary oo-ui-flaggedElement-progressive oo-ui-buttonWidget">

  171.             <button type="submit" class="oo-ui-inputWidget-input oo-ui-buttonElement-button">

  172.                 <span class="oo-ui-labelElement-label">Submit</span>

  173.             </button>

  174.         </span>

  175.     </div>

  176. </form>

  177. 

  178. % if result:

  179.     <div id="generation-time">

  180.         Results

  181.         % if result.cached:

  182.             <a id="cv-cached" href="#">cached<span>To save time (and money), this tool will retain the results of checks for up to 72 hours. This includes the URLs of the checked sources, but neither their content nor the content of the article. Future checks on the same page (assuming it remains unchanged) will not involve additional search queries, but a fresh comparison against the source URL will be made. If the page is modified, a new check will be run.</span></a> from <abbr title="${result.cache_time}">${result.cache_age} ago</abbr>. Originally

  183.         % endif

  184.         generated in <span class="mono">${round(result.time, 3)}</span>

  185.         % if query.action == "search":

  186.             seconds using <span class="mono">${result.queries}</span> quer${"y" if result.queries == 1 else "ies"}.

  187.         % else:

  188.             seconds.

  189.         % endif

  190.         <a href="${request.script_root | h}/?lang=${query.lang | h}&amp;project=${query.project | h}&amp;oldid=${query.oldid or query.page.lastrevid | h}&amp;action=${query.action | h}&amp;${"use_engine={0}&use_links={1}".format(int(query.use_engine not in ("0", "false")), int(query.use_links not in ("0", "false"))) if query.action == "search" else "" | h}${"url=" if query.action == "compare" else ""}${query.url if query.action == "compare" else "" | u}">Permalink.</a>

  191.     </div>

  192. 

  193.     <div id="cv-result" class="${'red' if result.confidence >= T_SUSPECT else 'yellow' if result.confidence >= T_POSSIBLE else 'green'}-box">

  194.         <table id="cv-result-head-table">

  195.             <colgroup>

  196.                 <col>

  197.                 <col>

  198.                 <col>

  199.             </colgroup>

  200.             <tr>

  201.                 <td>

  202.                     <a href="${query.page.url}">${query.page.title | h}</a>

  203.                     % if query.oldid:

  204.                         @<a href="https://${query.site.domain | h}/w/index.php?oldid=${query.oldid | h}">${query.oldid | h}</a>

  205.                     % endif

  206.                     % if query.redirected_from:

  207.                         <br />

  208.                         <span id="redirected-from">Redirected from <a href="https://${query.site.domain | h}/w/index.php?title=${query.redirected_from.title | u}&amp;redirect=no">${query.redirected_from.title | h}</a>. <a href="${request.url | httpsfix, h}&amp;noredirect=1">Check original.</a></span>

  209.                     % endif

  210.                 </td>

  211.                 <td>

  212.                     <div>

  213.                         % if result.confidence >= T_SUSPECT:

  214.                             Violation suspected

  215.                         % elif result.confidence >= T_POSSIBLE:

  216.                             Violation possible

  217.                         % elif result.sources:

  218.                             Violation unlikely

  219.                         % else:

  220.                             No violation

  221.                         % endif

  222.                     </div>

  223.                     <div>${round(result.confidence * 100, 1)}%</div>

  224.                     <div>similarity</div>

  225.                 </td>

  226.                 <td>

  227.                     % if result.url:

  228.                         <a href="${result.url | h}">${result.url | urlstrip, h}</a>

  229.                     % else:

  230.                         <span id="result-head-no-sources">No matches found.</span>

  231.                     % endif

  232.                 </td>

  233.             </tr>

  234.         </table>

  235.     </div>

  236. 

  237.     <% attrib = get_attribution_info(query.site, query.page) %>

  238.     % if attrib:

  239.         <div id="attribution-warning" class="yellow-box">

  240.             This article contains an attribution template: <code>{{<a href="${attrib[1]}">${attrib[0] | h}</a>}}</code>. Please verify that any potential copyvios are not from properly attributed sources.

  241.         </div>

  242.     % endif

  243. 

  244.     % if query.turnitin_result:

  245.         <div id="turnitin-container" class="${'red' if query.turnitin_result.reports else 'green'}-box">

  246.             <div id="turnitin-title">Turnitin Results</div>

  247.             % if query.turnitin_result.reports:

  248.                 <table id="turnitin-table"><tbody>

  249.                 % for report in turnitin_result.reports:

  250.                     <tr><td class="turnitin-table-cell"><a href="https://eranbot.toolforge.org/ithenticate.py?rid=${report.reportid}">Report ${report.reportid}</a> for text added at <a href="https://${query.lang}.wikipedia.org/w/index.php?title=${query.title}&amp;diff=${report.diffid}"> ${report.time_posted.strftime("%H:%M, %d %B %Y (UTC)")}</a>:

  251.                     <ul>

  252.                     % for source in report.sources:

  253.                           <li>${source['percent']}% of revision text (${source['words']} words) found at <a href="${source['url'] | h}">${source['url'] | h}</a></li>

  254.                     % endfor

  255.                     </ul></td></tr>

  256.                 % endfor

  257.                 </tbody></table>

  258.             % else:

  259.                 <div id="turnitin-summary">No matching sources found.</div>

  260.             % endif

  261.         </div>

  262.     % endif

  263. 

  264.     % if query.action == "search":

  265.         <% skips = False %>

  266.         <div id="sources-container">

  267.             <div id="sources-title">Checked Sources</div>

  268.             % if result.sources:

  269.                 <table id="cv-result-sources">

  270.                     <colgroup>

  271.                         <col>

  272.                         <col>

  273.                         <col>

  274.                     </colgroup>

  275.                     <tr>

  276.                         <th>URL</th>

  277.                         <th>Similarity</th>

  278.                         <th>Compare</th>

  279.                     </tr>

  280.                     % for i, source in enumerate(result.sources):

  281.                         <tr ${'class="source-default-hidden"' if i >= 10 else 'id="source-row-selected"' if i == 0 else ""}>

  282.                             <td><a ${'id="source-selected"' if i == 0 else ""} class="source-url" href="${source.url | h}">${source.url | h}</a></td>

  283.                             <td>

  284.                                 % if source.excluded:

  285.                                     <span class="source-excluded">Excluded</span>

  286.                                 % elif source.skipped:

  287.                                     <% skips = True %>

  288.                                     <span class="source-skipped">Skipped</span>

  289.                                 % else:

  290.                                     <span class="source-similarity ${"source-suspect" if source.confidence >= T_SUSPECT else "source-possible" if source.confidence >= T_POSSIBLE else "source-novio"}">${round(source.confidence * 100, 1)}%</span>

  291.                                 % endif

  292.                             </td>

  293.                             <td>

  294.                                 <a href="${request.script_root | h}/?lang=${query.lang | h}&amp;project=${query.project | h}&amp;oldid=${query.oldid or query.page.lastrevid | h}&amp;action=compare&amp;url=${source.url | u}">Compare</a>

  295.                             </td>

  296.                         </tr>

  297.                     % endfor

  298.                 </table>

  299.             % else:

  300.                 <div class="cv-source-footer">

  301.                     No sources checked.

  302.                 </div>

  303.             % endif

  304.             % if len(result.sources) > 10:

  305.                 <div id="cv-additional" class="cv-source-footer">

  306.                     ${len(result.sources) - 10} URL${"s" if len(result.sources) > 11 else ""} with lower similarity hidden. <a id="show-additional-sources" href="#">Show them.</a>

  307.                 </div>

  308.             % endif

  309.             % if skips or result.possible_miss:

  310.                 <div class="cv-source-footer">

  311.                     The search ended early because a match was found with high similarity. <a href="${request.url | httpsfix, h}&amp;noskip=1">Do a complete check.</a>

  312.                 </div>

  313.             % endif

  314.         </div>

  315.     % endif

  316.     <table id="cv-chain-table">

  317.         <tr>

  318.             <td class="cv-chain-cell">Article: <div class="cv-chain-detail"><p>${highlight_delta(result.article_chain, result.best.chains[1] if result.best else None)}</p></div></td>

  319.             <td class="cv-chain-cell">Source: <div class="cv-chain-detail"><p>${highlight_delta(result.best.chains[0], result.best.chains[1]) if result.best else ""}</p></div></td>

  320.         </tr>

  321.     </table>

  322. % endif

  323. <%include file="/support/footer.mako"/>