RUNOFF is a text markup language created in 1965 by J. E. Saltzer.
| #1399on PLDB | 59Years Old | 5Repos |
.na
.ll 72
.pl 90
.m1 4
.m2 4
.m3 6
.m4 6
.sp 8
.ds
.ce
CONTRIBUTING TO LINGUIST
.sp
.ce
by
.ce
GITHUB
.sp
.ce
and the
.sp
.ce
OPEN SOURCE COMMUNITY
.sp
.bp
.sp 5
.ce
_I_N_T_R_O_D_U_C_T_I_O_N:
.sp
Hi there! We're thrilled that you'd like to contribute to this
project. Your help is essential for keeping it great. This project
adheres to the Contributor Covenant Code of Conduct. By participating,
you are expected to uphold this code.
.br
The majority of contributions won't need to touch any Ruby code at all.
.sp 5
.ce
_A_d_d_i_n_g _a_n _e_x_t_e_n_s_i_o_n _t_o _a
_l_a_n_g_u_a_g_e
.sp
We try only to add new extensions once they have some usage on
GitHub. In most cases we prefer that extensions be in use in hundreds of
repositories before supporting them in Linguist.
.sp
To add support for a new extension:
.sp
.in 5
.un 5
1. Add your extension to the language entry in
_l_a_n_g_u_a_g_e_s_._y_m_l, keeping the extensions in
alphabetical order.
.br
.un 5
2. Add at least one sample for your extension to the samples directory
in the correct subdirectory.
.br
.un 5
3. Open a pull request, linking to a GitHub search result showing
in-the-wild usage.
.in 0
.sp
In addition, if this extension is already listed in
_l_a_n_g_u_a_g_e_s_._y_m_l then sometimes a few more steps
will need to be taken:
.sp
.in 5
.un 5
1. Make sure that example .yourextension files are present in the
samples directory for each language that uses .yourextension.
.br
.un 5
2. Test the performance of the Bayesian classifier with a relatively
large number (1000s) of sample .yourextension files. (ping @arfon or
@bkeepers to help with this) to ensure we're not misclassifying files.
.br
.un 5
3. If the Bayesian classifier does a bad job with the sample files
then a heuristic may need to be written to help.
.in 0
.sp 5
.ce
_A_d_d_i_n_g _a _l_a_n_g_u_a_g_e
.sp
We try only to add languages once they have some usage on GitHub.
In most cases we prefer that each new extension be in use in hundreds of
repositories before supporting them in Linguist.
.sp
To add support for a new language:
.in 5
.un 5
1. Add an entry for your language to
_l_a_n_g_u_a_g_e_s_._y_m_l.
.br
.un 5
2. Add a grammar for your language. Please only add grammars that have
a license that permits redistribution.
.br
.in +5
.un 5
i. Add your grammar as a submodule:
.br
.in +4
git submodule add
https://github.com/Alhadis/language-roff
vendor/grammars/language-roff
.in -4
.un 5
ii. Add your grammar to grammars.yml:
.br
.in +4
script/convert-grammars --add vendor/grammars/MyGrammar
.in -4
.un 5
iii. Download the license for the grammar by running script/licensed.
Be careful to only commit the file for the new grammar, as this script
may update licenses for other grammars as well.
.br
.in -5
.un 5
3. Add samples for your language to the samples directory in the
correct subdirectory.
.br
.un 5
4. Open a pull request, linking to a GitHub search result showing
in-the-wild usage.
.br
.in 0
.sp
In addition, if your new language defines an extension that's already
listed in _l_a_n_g_u_a_g_e_s_._y_m_l (such as `.foo`) then
sometimes a few more steps will need to be taken:
.sp
.in +5
.un 5
1. Make sure that example .foo files are present in the samples
directory for each language that uses .foo.
.br
.un 5
2. Test the performance of the Bayesian classifier with a relatively
large number (1000s) of sample `.foo` files. (ping @arfon or @bkeepers
to help with this) to ensure we're not misclassifying files.
.br
.un 5
3. If the Bayesian classifier does a bad job with the sample .foo
files then a heuristic may need to be written to help.
.br
.in 0
.sp
Remember, the goal here is to try and avoid false positives!
.sp 2
.ce
_F_i_x_i_n_g _a _m_i_s_c_l_a_s_s_i_f_i_e_d
_l_a_n_g_u_a_g_e
.br
Most languages are detected by their file extension defined in
_l_a_n_g_u_a_g_e_s_._y_m_l. For disambiguating between
files with common extensions, linguist applies some heuristics and a
statistical classifier. This process can help differentiate between,
for example, .h files which could be either C, C++, or Obj-C.
.sp
Misclassifications can often be solved by either adding a new
filename or extension for the language or adding more samples to make
the classifier smarter.
.sp
.m4 -2
.ce
_F_i_x_i_n_g _s_y_n_t_a_x
_h_i_g_h_l_i_g_h_t_i_n_g
.br
Syntax highlighting in GitHub is performed using
TextMate-compatible grammars. These are the same grammars that TextMate,
Sublime Text and Atom use. Every language in languages.yml is mapped to
its corresponding TM `scope`. This scope will be used when picking up a
grammar for highlighting.
.sp
Assuming your code is being detected as the right language, in most
cases this is due to a bug in the language grammar rather than a bug in
Linguist. _g_r_a_m_m_a_r_s_._y_m_l lists all the grammars
we use for syntax highlighting on github.com. Find the one corresponding
to your code's programming language and submit a bug report upstream.
.sp
If you can, try to reproduce the highlighting problem in the text editor
that the grammar is designed for (TextMate, Sublime Text, or Atom) and
include that information in your bug report.
.sp
You can also try to fix the bug yourself and submit a Pull Request.
TextMate's documentation offers a good introduction on how to work with
TextMate-compatible grammars. You can test grammars using Lightshow.
.sp
Once the bug has been fixed upstream, we'll pick it up for GitHub
in the next release of Linguist.
.sp 2
.ce
_T_e_s_t_i_n_g
.br
For development you are going to want to checkout out the source.
To get it, clone the repo and run Bundler to install its dependencies.
.sp
.in 4
git clone https://github.com/github/linguist.git
.br
cd linguist/
.br
script/bootstrap
.br
.in 0
.sp
To run the tests:
.sp
.in 4
bundle exec rake test
.in 0
.sp
Sometimes getting the tests running can be too much work, especially
if you don't have much Ruby experience. It's okay: be lazy and let our
build bot Travis run the tests for you. Just open a pull request and the
bot will start cranking away.
.sp
.ce
_M_a_i_n_t_a_i_n_e_r_s
.br
Linguist is maintained with love by:
.sp
.in -2
- @arfon (GitHub Staff)
.br
- @larsbrinkhoff
.br
- @pchaigno
.in 0
.br
.sp
As Linguist is a production dependency for GitHub we have a couple of
workflow restrictions:
.sp
.in -2
- Anyone with commit rights can merge Pull Requests provided that there
is a :+1: from a GitHub member of staff
.br
- Releases are performed by GitHub staff so we can ensure GitHub.com
always stays up to date with the latest release of Linguist and there
are no regressions in production.
.in 0
.sp
.ce
_R_e_l_e_a_s_i_n_g
.sp
If you are the current maintainer of this gem:
.sp
.in 5
.ul 5
1. Create a branch for the release:
.sp
.in +2
git checkout -b cut-release-vxx.xx.xx
.in -2
.sp
.ul 5
2. Make sure your local dependencies are up to date:
.sp
.in +2
script/bootstrap
.in -2
.sp
.ul 5
3. If grammar submodules have not been updated recently, update them:
.sp
.in +2
git submodule update --remote _&_& git commit -a
.in -2
.sp
.ul 5
4. Ensure that samples are updated:
.sp
.in +2
bundle exec rake samples
.in -2
.sp
5. Ensure that tests are green:
.sp
.in +2
bundle exec rake test
.in -2
.sp
.ul 5
6. Bump gem version in lib/linguist/version.rb
.br
.ul 5
7. Make a PR to github/linguist
.br
.ul 5
8. Build a local gem: `bundle exec rake build_gem`
.br
.ul 5
9. Test the gem:
.sp
.in +5
.un 5
i. Bump the Gemfile and Gemfile.lock versions for an app which relies
on this gem
.un 5
ii. Install the new gem locally
.un 5
iii. Test behaviour locally, branch deploy, whatever needs to happen.
.br
.in -5
.sp
.ul 5
10. Merge github/linguist PR
.sp
.ul 5
11. Tag and push:
.sp
.in +2
git tag vx.xx.xx;
.br
git push --tags
.in -2
.sp
12. Push to rubygems.org
.br
.in +2
gem push github-linguist-3.0.0.gem
.in -2
.sp 2