Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Id3 #47

Merged
merged 9 commits into from
Oct 27, 2019
Merged

Id3 #47

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ Jerome Charaoui
### Fixed
- Remove the erroneous error messages when the user supplies wrong command line
options.
- The same cache folder is used, irrespective whether the server root URL ends
with '/'

## [1.1.10] - 2019-09-10
### Added
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
VERSION=1.2.0

CFLAGS += -O2 -Wall -Wextra -Wshadow -rdynamic -D_GNU_SOURCE\
CFLAGS += -g -O2 -Wall -Wextra -Wshadow -rdynamic -D_GNU_SOURCE\
-D_FILE_OFFSET_BITS=64 -DVERSION=\"$(VERSION)\"\
`pkg-config --cflags-only-I gumbo libcurl fuse uuid expat`
LIBS = -pthread -lgumbo -lcurl -lfuse -lcrypto -luuid -lexpat
Expand Down
40 changes: 35 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,14 @@ HTTPDirFS options:
--retry-wait Set delay in seconds before retrying an HTTP request
after encountering an error. (default: 5)
--user-agent Set user agent string (default: "HTTPDirFS")
--no-range-check Disable the build-in check for the server's support
for HTTP range requests

Subsonic options:
For mounting a Airsonic / Subsonic server:
--sonic-username The username for your Airsonic / Subsonic server
--sonic-password The username for your Airsonic / Subsonic server
--sonic-id3 Enable ID3 mode - this present the server content in
Artist/Album/Song layout

FUSE options:

Expand Down Expand Up @@ -92,15 +96,32 @@ please refer to

## Airsonic / Subsonic server support
This is a new feature to 1.2.0. Now you can mount the music collection on your
Airsonic / Subsonic server, and browse them using your favourite file browser.
Airsonic / Subsonic server (*sonic), and browse them using your favourite file browser.
You simply have to supply both ``--sonic-username`` and ``--sonic-password`` to
trigger the Airsonic / Subsonic server mode. For example:
trigger the *sonic server mode. For example:

./httpdirfs -f --cache --sonic-username $USERNAME --sonic-password $PASSWORD $URL $MOUNT_POINT

You definitely want to enable the cache for this one, otherwise it is painfully
slow.

There are two ways of mounting your *sonic server
- the index mode
- and the ID3 mode.

In the index mode, the filesystem is presented based on the listing on the
``Index`` link in your *sonic's home page.

In ID3 mode, the filesystem is presented using the following hierarchy:
0. Root
1. Alphabetical indices of the Artists' name
2. The Arists' name
3. All of the albums by a single artist
4. All the songs in an album.

By default, *sonic server is mounted in the Index mode. If you want to mount in
ID3 mode, please use the ``--sonic-id3`` flag.

## Configuration file support
This program has basic support for using a configuration file. The configuration
file that the program reads is ``${XDG_CONFIG_HOME}/httpdirfs/config``, which by
Expand Down Expand Up @@ -143,15 +164,24 @@ enable them, compile the program with the ``-DCACHE_LOCK_DEBUG``, the
make CPPFLAGS=-DCACHE_LOCK_DEBUG

## The Technical Details
This program downloads the HTML web pages/files using
[libcurl](https://curl.haxx.se/libcurl/), then parses the listing pages using
For the normal HTTP directories, this program downloads the HTML web pages/files
using [libcurl](https://curl.haxx.se/libcurl/), then parses the listing pages using
[Gumbo](https://github.com/google/gumbo-parser), and presents them using
[libfuse](https://github.com/libfuse/libfuse).

For *sonic servers, rather than using the Gumbo parser, this program parse
*sonic servers' XML responses using
[expat](https://github.com/libexpat/libexpat).

The cache system stores the metadata and the downloaded file into two
separate directories. It uses ``uint8_t`` arrays to record which segments of the
file had been downloaded.

Note that HTTPDirFS requires the server to support HTTP Range Request, some
servers support this features, but does not present ``"Accept-Ranges: bytes`` in
the header responses. HTTPDirFS by default checks for this header field. You can
disable this check by using the ``--no-range-check`` flag.

## Other projects which incorporate HTTPDirFS
- [Curious Container](https://www.curious-containers.cc/docs/red-connector-http#mount-dir)
has a Python wrapper for mounting HTTPDirFS.
Expand Down
8 changes: 4 additions & 4 deletions src/cache.c
Original file line number Diff line number Diff line change
Expand Up @@ -585,7 +585,7 @@ void Cache_delete(const char *fn)
{
if (CONFIG.sonic_mode) {
Link *link = path_to_Link(fn);
fn = link->sonic_id_str;
fn = link->sonic_song_id_str;
}

char *metafn = path_append(META_DIR, fn);
Expand Down Expand Up @@ -678,7 +678,7 @@ int Cache_create(const char *path)
fn = curl_easy_unescape(NULL, this_link->f_url + ROOT_LINK_OFFSET, 0,
NULL);
} else {
fn = this_link->sonic_id_str;
fn = this_link->sonic_song_id_str;
}
fprintf(stderr, "Cache_create(): Creating cache files for %s.\n", fn);

Expand Down Expand Up @@ -774,7 +774,7 @@ Cache *Cache_open(const char *fn)
return NULL;
}
} else {
if (!Cache_exist(link->sonic_id_str)) {
if (!Cache_exist(link->sonic_song_id_str)) {
return NULL;
}
}
Expand All @@ -788,7 +788,7 @@ Cache *Cache_open(const char *fn)

/* Set the path for the local cache file, if we are in sonic mode */
if (CONFIG.sonic_mode) {
fn = link->sonic_id_str;
fn = link->sonic_song_id_str;
}

cf->path = strndup(fn, MAX_PATH_LEN);
Expand Down
71 changes: 45 additions & 26 deletions src/link.c
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

#include <gumbo.h>

#include <assert.h>
#include <ctype.h>
#include <errno.h>
#include <stdlib.h>
Expand All @@ -27,8 +28,15 @@ int ROOT_LINK_OFFSET = 0;
*/
static pthread_mutex_t link_lock;

LinkTable *LinkSystem_init(const char *url)
LinkTable *LinkSystem_init(const char *raw_url)
{
/* Remove excess '/' if it is there */
char *url = strdup(raw_url);
int url_len = strnlen(url, MAX_PATH_LEN) - 1;
if (url[url_len] == '/') {
url[url_len] = '\0';
}

if (pthread_mutex_init(&link_lock, NULL) != 0) {
fprintf(stderr,
"link_system_init(): link_lock initialisation failed!\n");
Expand All @@ -37,17 +45,7 @@ LinkTable *LinkSystem_init(const char *url)

/* --------- Set the length of the root link ----------- */
/* This is where the '/' should be */
ROOT_LINK_OFFSET = strnlen(url, MAX_PATH_LEN) - 1;
if (url[ROOT_LINK_OFFSET] != '/') {
/*
* If '/' is not there, it is automatically added, so we need to skip 2
* characters
*/
ROOT_LINK_OFFSET += 2;
} else {
/* If '/' is there, we need to skip it */
ROOT_LINK_OFFSET += 1;
}
ROOT_LINK_OFFSET = strnlen(url, MAX_PATH_LEN) + 1;

/* --------------------- Enable cache system -------------------- /
*
Expand All @@ -67,7 +65,11 @@ LinkTable *LinkSystem_init(const char *url)
ROOT_LINK_TBL = LinkTable_new(url);
} else {
sonic_config_init(url, CONFIG.sonic_username, CONFIG.sonic_password);
ROOT_LINK_TBL = sonic_LinkTable_new(0);
if (!CONFIG.sonic_id3) {
ROOT_LINK_TBL = sonic_LinkTable_new_index(0);
} else {
ROOT_LINK_TBL = sonic_LinkTable_new_id3(0, 0);
}
}
return ROOT_LINK_TBL;
}
Expand Down Expand Up @@ -400,6 +402,7 @@ LinkTable *LinkTable_alloc(const char *url)
Link *head_link = Link_new("/", LINK_HEAD);
LinkTable_add(linktbl, head_link);
strncpy(head_link->f_url, url, MAX_PATH_LEN);
assert(linktbl->num == 1);
return linktbl;
}

Expand Down Expand Up @@ -597,14 +600,21 @@ LinkTable *LinkTable_disk_open(const char *dirn)
LinkTable *path_to_Link_LinkTable_new(const char *path)
{
Link *link = path_to_Link(path);
if (!link->next_table) {
LinkTable *next_table = link->next_table;
if (!next_table) {
if (!CONFIG.sonic_mode) {
link->next_table = LinkTable_new(link->f_url);
next_table = LinkTable_new(link->f_url);
} else {
link->next_table = sonic_LinkTable_new(link->sonic_id);
if (!CONFIG.sonic_id3) {
next_table = sonic_LinkTable_new_index(link->sonic_id);
} else {
next_table = sonic_LinkTable_new_id3(link->sonic_depth,
link->sonic_id);
}
}
}
return link->next_table;
link->next_table = next_table;
return next_table;
}

static Link *path_to_Link_recursive(char *path, LinkTable *linktbl)
Expand Down Expand Up @@ -645,17 +655,24 @@ static Link *path_to_Link_recursive(char *path, LinkTable *linktbl)
for (int i = 1; i < linktbl->num; i++) {
if (!strncmp(path, linktbl->links[i]->linkname, MAX_FILENAME_LEN)) {
/* The next sub-directory exists */
if (!linktbl->links[i]->next_table) {
LinkTable *next_table = linktbl->links[i]->next_table;
if (!next_table) {
if (!CONFIG.sonic_mode) {
linktbl->links[i]->next_table = LinkTable_new(
next_table = LinkTable_new(
linktbl->links[i]->f_url);
} else {
linktbl->links[i]->next_table = sonic_LinkTable_new(
linktbl->links[i]->sonic_id);
if (!CONFIG.sonic_id3) {
next_table = sonic_LinkTable_new_index(
linktbl->links[i]->sonic_id);
} else {
next_table = sonic_LinkTable_new_id3(
linktbl->links[i]->sonic_depth,
linktbl->links[i]->sonic_id);
}
}
}
return path_to_Link_recursive(
next_path, linktbl->links[i]->next_table);
linktbl->links[i]->next_table = next_table;
return path_to_Link_recursive(next_path, next_table);
}
}
}
Expand Down Expand Up @@ -718,10 +735,12 @@ long path_download(const char *path, char *output_buf, size_t size,
transfer_blocking(curl);

/* Check for range seek support */
if (!strcasestr((header.data), "Accept-Ranges: bytes")) {
fprintf(stderr, "Error: This web server does not support HTTP \
if (!CONFIG.no_range_check) {
if (!strcasestr((header.data), "Accept-Ranges: bytes")) {
fprintf(stderr, "Error: This web server does not support HTTP \
range requests\n");
exit(EXIT_FAILURE);
exit(EXIT_FAILURE);
}
}

free(header.data);
Expand Down
21 changes: 16 additions & 5 deletions src/link.h
Original file line number Diff line number Diff line change
Expand Up @@ -69,14 +69,25 @@ struct Link {
/** \brief The pointer associated with the cache file */
Cache *cache_ptr;
/**
* \brief Sonic Music Directory ID
* \details We use linkname to store filename
* \brief Sonic id field
* \details This is used to store the followings:
* - Arist ID
* - Album ID
* - Song ID
* - Sub-directory ID (in the XML response, this is the ID on the "child"
* element)
*/
int sonic_id;
/**
* \brief Sonic Music Directory ID in string format
* \brief Sonic directory depth
* \details This is used exclusively in ID3 mode to store the depth of the
* current directory.
*/
char *sonic_id_str;
int sonic_depth;
/**
* \brief The sonic song's ID in character array format.
*/
char *sonic_song_id_str;
};

struct LinkTable {
Expand All @@ -97,7 +108,7 @@ extern int ROOT_LINK_OFFSET;
/**
* \brief initialise link sub-system.
*/
LinkTable *LinkSystem_init(const char *url);
LinkTable *LinkSystem_init(const char *raw_url);

/**
* \brief Add a link to the curl multi bundle for querying stats
Expand Down
14 changes: 13 additions & 1 deletion src/main.c
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ int main(int argc, char **argv)
CONFIG.sonic_mode = 1;
} else if (CONFIG.sonic_username || CONFIG.sonic_password) {
fprintf(stderr,
"Error: You have to supply both username and password to\
"Error: You have to supply both username and password to \
activate Sonic mode.\n");
exit(EXIT_FAILURE);
}
Expand Down Expand Up @@ -151,6 +151,8 @@ parse_arg_list(int argc, char **argv, char ***fuse_argv, int *fuse_argc)
{"cache-location", required_argument, NULL, 'L'}, /* 14 */
{"sonic-username", required_argument, NULL, 'L'}, /* 15 */
{"sonic-password", required_argument, NULL, 'L'}, /* 16 */
{"sonic-id3", no_argument, NULL, 'L'}, /* 17 */
{"no-range-check", no_argument, NULL, 'L'}, /* 18 */
{0, 0, 0, 0}
};
while ((c =
Expand Down Expand Up @@ -224,6 +226,12 @@ parse_arg_list(int argc, char **argv, char ***fuse_argv, int *fuse_argc)
case 16:
CONFIG.sonic_password = strdup(optarg);
break;
case 17:
CONFIG.sonic_id3 = 1;
break;
case 18:
CONFIG.no_range_check = 1;
break;
default:
fprintf(stderr, "see httpdirfs -h for usage\n");
return 1;
Expand Down Expand Up @@ -296,9 +304,13 @@ HTTPDirFS options:\n\
--retry-wait Set delay in seconds before retrying an HTTP request\n\
after encountering an error. (default: 5)\n\
--user-agent Set user agent string (default: \"HTTPDirFS\")\n\
--no-range-check Disable the build-in check for the server's support\n\
for HTTP range requests\n\
\n\
For mounting a Airsonic / Subsonic server:\n\
--sonic-username The username for your Airsonic / Subsonic server\n\
--sonic-password The username for your Airsonic / Subsonic server\n\
--sonic-id3 Enable ID3 mode - this present the server content in\n\
Artist/Album/Song layout \n\
\n");
}
Loading