|
<!DOCTYPE html> |
|
<html lang="en"> |
|
<head> |
|
<meta charset="UTF-8"> |
|
<meta name="viewport" content="width=device-width, initial-scale=1.0"> |
|
<title>Stick To Your Role! About</title> |
|
|
|
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/5.1.3/css/bootstrap.min.css"> |
|
|
|
<link rel="stylesheet" href="https://cdn.datatables.net/1.11.5/css/dataTables.bootstrap5.min.css"> |
|
|
|
<style> |
|
body { |
|
background-color: #f8f9fa; |
|
font-family: 'Arial', sans-serif; |
|
} |
|
.container { |
|
max-width: 1200px; |
|
margin: auto; |
|
padding: 20px; |
|
background: #fff; |
|
border-radius: 8px; |
|
box-shadow: 0 4px 8px rgba(0,0,0,0.1); |
|
} |
|
h1 { |
|
color: #333; |
|
text-align: center; |
|
} |
|
h2 { |
|
color: #333; |
|
margin-top: 30px; |
|
text-align: center; |
|
} |
|
|
|
.section { |
|
padding-top: 19px; |
|
text-align: left; |
|
} |
|
|
|
.section p { |
|
padding-left: 150px; |
|
padding-right: 150px; |
|
text-indent: 2em; |
|
margin: auto; |
|
margin-bottom: 10px; |
|
text-align: left; |
|
} |
|
|
|
.section ol, ul { |
|
padding-left: 150px; |
|
padding-right: 150px; |
|
margin: auto; |
|
margin-bottom: 20px; |
|
margin-left: 50px; |
|
text-align: left; |
|
margin-top: 0px; |
|
} |
|
|
|
.citation-section { |
|
width: 100%; |
|
margin-top: 50px; |
|
text-align: center; |
|
} |
|
.citation-box { |
|
background-color: #f8f9fa; |
|
border: 1px solid #dee2e6; |
|
border-radius: 8px; |
|
padding: 10px; |
|
margin-top: 5px; |
|
font-size: 15px; |
|
text-align: left; |
|
font-family: 'Courier New', Courier, monospace; |
|
white-space: pre; |
|
} |
|
|
|
.image-container-structure { |
|
display: flex; |
|
justify-content: center; |
|
gap: 10px; |
|
margin-bottom: 40px; |
|
max-width: 70%; |
|
margin: auto; |
|
} |
|
|
|
.image-container-structure a { |
|
flex: 1; |
|
} |
|
|
|
.image-container-structure img { |
|
max-width: 100%; |
|
height: auto; |
|
display: block; |
|
margin: auto; |
|
} |
|
|
|
.image-container { |
|
width: 100%; |
|
margin-bottom: 40px; |
|
} |
|
.image-container #admin-questionnaire { |
|
width: 50%; |
|
height: auto; |
|
display: block; |
|
margin: auto; |
|
} |
|
.image-container #ro-image { |
|
width: 70%; |
|
height: auto; |
|
display: block; |
|
margin: auto; |
|
} |
|
|
|
.section-title { |
|
font-size: 24px; |
|
font-weight: bold; |
|
text-align: center; |
|
margin-bottom: 40px; |
|
padding: 20px; |
|
background-color: #610b5d; |
|
color: #fff; |
|
border-radius: 15px; |
|
} |
|
.back-button { |
|
text-align: center; |
|
margin-top: 50px; |
|
} |
|
.custom-button { |
|
background-color: #610b5d; |
|
color: #fff; |
|
border-radius: 15px; |
|
padding: 10px 20px; |
|
font-size: 18px; |
|
text-decoration: none; |
|
} |
|
.custom-button:hover { |
|
background-color: #812b7d; |
|
color: #fff; |
|
} |
|
.form-container { |
|
max-width: 80%; |
|
margin: 20px 100px; |
|
padding: 50px 150px; |
|
text-align: center; |
|
background-color: #f8f9fa; |
|
} |
|
|
|
.form-row { |
|
max-width: 100%; |
|
margin-bottom: 20px; |
|
text-align: left; |
|
} |
|
|
|
.form-label { |
|
} |
|
|
|
.col-md-4 { |
|
width: 100% |
|
} |
|
.col-md-8 { |
|
width: 100% |
|
} |
|
|
|
.form-content { |
|
margin-bottom: 15px; |
|
min-width: 100%; |
|
} |
|
.form-content::placeholder { |
|
color: #aaa; |
|
font-style: italic; |
|
} |
|
|
|
.file-input { |
|
margin-top: 10px; |
|
} |
|
</style> |
|
</head> |
|
<body> |
|
<div class="container"> |
|
<h1 class="mt-5">Stick To Your Role! Leaderboard</h1> |
|
<div class="table-responsive"> |
|
|
|
{{ table_html|safe }} |
|
</div> |
|
<div class="section"> |
|
<div id="evaluate_custom_model" class="section-title">Evaluate a custom model</div> |
|
<p> |
|
To evaluate a custom model you can use our <a href="https://gitlab.inria.fr/gkovac/value_stability">open-source code</a>. |
|
If a model is in the huggingface transformers format (saved either localy or on the hub), |
|
it can be simply added by adding a config file. |
|
The model can then be evaluated as any other model. |
|
To do so, follow the <a href="https://gitlab.inria.fr/gkovac/value_stability/-/blob/master/README.md?ref_type=heads#adding-a-new-model">instructions</a> in the README.md file. |
|
</p> |
|
</div> |
|
<div class="section" id="paper"> |
|
<div class="section-title">Submit a custom model to the Stick To Your Role! Leaderboard</div> |
|
<p> |
|
If you want, your model can be to the Stick To Your Role! Leaderboard, as an unofficial submission. |
|
A separate list of models containing both official and unofficial submissions will be created. |
|
The procedure is as follows: |
|
</p> |
|
<ol> |
|
<li> |
|
<b> Add and evaluate your model </b> - Add your model as a config file as described <a href="{{ url_for('new_model', _anchor='evaluate_custom_model') }}">above</a>. |
|
This procedure should result in 9 json files as such: |
|
<code>`Leaderboard/results/stability_leaderboard/<your_model_name>/chunk_0_<timestamp>/results.json`</code> |
|
</li> |
|
<li> |
|
<b> Submit the config file </b> - Create a pull request to our <a href="https://gitlab.inria.fr/gkovac/value_stability">repository</a> from a branch <code>"unofficial_model/<your_model_name>"</code>. |
|
The pull request should ideally only add the config file in <code>`./models/leaderboard_configs`</code>. |
|
If additional changes are needed, they should ideally be constrained to a new model class (see <a href="https://gitlab.inria.fr/gkovac/value_stability/-/blob/master/models/huggingfacemodel.py?ref_type=heads">huggingfacemodel.py</a> for reference). |
|
<li> |
|
<b> Submit the model results </b> - submit the *json files as a ZIP using the form below. |
|
We will integrate the model's results on our side, and rerank models with yours included. |
|
</li> |
|
</ol> |
|
<div class="form-container"> |
|
<form id="model-submission-form" method="POST" action="{{ url_for('submit_model') }}" enctype="multipart/form-data"> |
|
<div class="form-row row"> |
|
<div class="col-md-4"> |
|
<label for="model_name" class="form-label">Model Name:</label> |
|
</div> |
|
<div class="col-md-8"> |
|
<input type="text" class="form-content" id="model_name" name="model_name" required> |
|
</div> |
|
</div> |
|
<div class="form-row row"> |
|
<div class="col-md-4"> |
|
<label for="pull_request_link" class="form-label">Pull Request Link:</label> |
|
</div> |
|
<div class="col-md-8"> |
|
<input type="url" class="form-content" id="pull_request_link" name="pull_request_link" required> |
|
</div> |
|
</div> |
|
<div class="form-row row"> |
|
<div class="col-md-4"> |
|
<label for="email" class="form-label">Email:</label> |
|
</div> |
|
<div class="col-md-8"> |
|
<input type="email" class="form-content" id="email" name="email" required> |
|
</div> |
|
</div> |
|
<div class="form-row row"> |
|
<div class="col-md-4"> |
|
<label for="description" class="form-label">Description:</label> |
|
</div> |
|
<div class="col-md-8"> |
|
<textarea class="form-content" id="description" name="description" placeholder="Various details on the model training and architecture (e.g. dataset, model size, optimizer, etc.)" rows="3" required></textarea> |
|
</div> |
|
</div> |
|
<div class="form-row row"> |
|
<div class="col-md-4"> |
|
<label for="model_files" class="form-label"> |
|
Upload the Model results directory as a ZIP file |
|
(<code>Leaderboard/results/stability_leaderboard/<your_model_name></code>): |
|
</label> |
|
</div> |
|
<div class="col-md-8"> |
|
<input type="file" id="model_files" name="model_files" class="file-input" accept=".zip" required> |
|
<small class="form-text text-muted"> |
|
Please upload a ZIP file containing the results directory. |
|
</small> |
|
</div> |
|
</div> |
|
<button type="submit" class="btn custom-button mt-3">Submit</button> |
|
</form> |
|
</div> |
|
</div> |
|
<div class="back-button"> |
|
<a href="{{ url_for('index') }}" class="custom-button mt-3">Main page</a> |
|
</div> |
|
</div> |
|
|
|
|
|
<script src="https://code.jquery.com/jquery-3.6.0.min.js"></script> |
|
|
|
<script src="https://stackpath.bootstrapcdn.com/bootstrap/5.1.3/js/bootstrap.bundle.min.js"></script> |
|
|
|
<script src="https://cdn.datatables.net/1.11.5/js/jquery.dataTables.min.js"></script> |
|
<script src="https://cdn.datatables.net/1.11.5/js/dataTables.bootstrap5.min.js"></script> |
|
|
|
<script> |
|
$(document).ready(function() { |
|
const table = $('table').DataTable({ |
|
"paging": false, |
|
"info": false, |
|
"columnDefs": [ |
|
{ "orderable": false, "targets": 0 }, |
|
{ "searchable": false, "targets": 0 } |
|
], |
|
"order": [[ 2, 'desc' ]], |
|
"drawCallback": function(settings) { |
|
var api = this.api(); |
|
api.column(0, {order:'applied'}).nodes().each(function(cell, i) { |
|
cell.innerHTML = i + 1; |
|
}); |
|
} |
|
}); |
|
}); |
|
</script> |
|
</body> |
|
</html> |
|
|
